From: Thomas Walker Lynch Date: Sat, 8 Aug 2026 15:28:59 +0000 (+0000) Subject: . X-Git-Url: https://git.reasoningtechnology.com/%27%20%20%20window.RT.dirpr_library%20%20%20%27/%27%20%20%20key%20%20%20%27?a=commitdiff_plain;h=1919fe17c747bc641a5a10e42a929bac90aee6ea;p=TM-2026 . --- diff --git a/document/book/TM-2026.html b/document/book/TM-2026.html index 4240f70..0e78bac 100644 --- a/document/book/TM-2026.html +++ b/document/book/TM-2026.html @@ -219,16 +219,19 @@ The Turing Machine transforms mathematics into computing -

- To apply his proof to the Entscheidungsproblem, Turing carried the additional burden of establishing that Hilbert and Ackermann's intuitive concept of an effective procedure was functionally equivalent to a Turing Machine program. Alonzo Church had already made an identification of this kind, some months earlier, defining effective calculability by identifying it with general recursiveness, or equivalently with λ-definability in his own lambda calculus Alonzo Church, "An Unsolvable Problem of Elementary Number Theory," American Journal of Mathematics 58, no. 2 (April 1936): 356. A preliminary statement was presented to the American Mathematical Society in April 1935. The name 'Church's thesis' is due to Kleene.. What Church gave was a definition, an identification of one formal notion with another. What Turing gave was an argument: he analyzed what a person does when he computes with pencil and paper, and then built a machine to match. Church acknowledged the difference in his 1937 review of Turing's paper, the review in which he also coined the phrase 'Turing machine', granting that computability by such a machine had the advantage of "making the identification with effectiveness in the ordinary (not explicitly defined) sense evident immediately", without preliminary theorems Alonzo Church, review of A. M. Turing, "On Computable Numbers, with an Application to the Entscheidungsproblem," The Journal of Symbolic Logic 2, no. 1 (March 1937): 42–43.. Gödel, who had found Church's proposal unsatisfactory, accepted the identification only upon seeing Turing's. -

+

+ To apply his proof to the Entscheidungsproblem, Turing carried the additional burden of establishing that Hilbert and Ackermann's intuitive concept of an effective procedure was functionally equivalent to a Turing Machine program. Alonzo Church had made an identification of this kind some months earlier + Alonzo Church, "An Unsolvable Problem of Elementary Number Theory," American Journal of Mathematics 58, no. 2 (April 1936): 345–363, the identification at 356. A preliminary statement was presented to the American Mathematical Society on 1935-04-19. The name 'Church's thesis' is due to Kleene. Church first framed the identification in terms of his own λ-definability. Gödel told him that this was thoroughly unsatisfactory, and Church restated it in terms of Herbrand-Gödel general recursiveness, which is the form that reached print. The exchange survives in a letter from Church to Kleene of 1935-11-29, quoted in Stephen C. Kleene, "Origins of Recursive Function Theory," Annals of the History of Computing 3, no. 1 (1981): 52–67, at 59., + and there was some controversy over his proposal. Turing then provided his naturalist argument, based on what a person does when he computes with pencil and paper. This made the explanation more intuitive, and it resolved the controversy + Church himself conceded that the 'Turing machine' explanation had the advantage of "making the identification with effectiveness in the ordinary (not explicitly defined) sense evident immediately", without preliminary theorems. Alonzo Church, review of A. M. Turing, "On Computable Numbers, with an Application to the Entscheidungsproblem," The Journal of Symbolic Logic 2, no. 1 (March 1937): 42–43. It runs two pages, and is the clearest short statement of the difference, made by the man who stood to lose by it. Gödel, who had rejected Church's proposal, came to the same view, remarking to Hao Wang that the sharp concept of a mechanical procedure was not perceived before Turing, who brought us to the right perspective; Hao Wang, From Mathematics to Philosophy (London: Routledge and Kegan Paul, 1974), 85. How far Gödel ever accepted the thesis in Church's own form is disputed; see Martin Davis, "Why Gödel Didn't Have Church's Thesis," Information and Control 54 (1982): 3–24.. +

Over the following decades, the academic community evaluated and accepted the argument, cementing what is now known as the Church-Turing Thesis. This consensus supplied the necessary bridge between mathematics and modern computer science by formally equating the vague, historical notion of a human procedure with the rigorous, mechanical definition of an algorithm.

- For Turing's purposes working on the Entscheidungsproblem, establishing functional equivalence between algorithms and Turing Machine programs was sufficient. However, when the Turing Machine serves as a foundational model for computation theory, we are led to ask another question: whether the Turing Machine is representative of modern architectures, and to the extent it differs, how this would affect the applicability of computation theoretic results. + For Turing's purposes working on the Entscheidungsproblem, establishing functional equivalence between algorithms and Turing Machine programs was sufficient. However, when the Turing Machine serves as a foundational model for computation theory, we are led to ask another question: whether the Turing Machine is representative of modern architectures, and to the extent it differs, how this would affect the applicability of computation theoretic results. That question occupies the remainder of this chapter, and the answer arrived at is that neither side is quite fit to be compared to the other as it stands.

@@ -236,30 +239,36 @@

- Furthermore Turing restricted the figures printed by his a-machine to 0 and 1, so that the sequences it computes are binary. George Boole's work (1847, 1854) was well established by then, so from a theoretical standpoint, it was a sensible simplification. However, utilizing binary within the context of a machine description effectively bridged the gap to the more practically minded engineers of the time. Alan Turing's paper arrived at the same time that switched telephone networks had reached a scale that made them difficult to maintain without systematic approaches. These networks were built upon electromechanical relays, which were decisively binary devices. At least seven men in addition to Alan Turing appear to have independently contemplated the intersection of Boolean algebra, logic, and physical computing: Victor Shestakov (1935, proposed mapping Boolean algebra to electromechanical relay circuits), Konrad Zuse (1936, adopted base 2 architecture to bypass the physical complexity of decimal mechanical gears), Akira Nakashima (1936, published the mathematical equivalence of Boolean algebra and two-terminal switching networks), Louis Couffignal (1936, argued calculating machines must shift to binary linkages to reduce physical friction), Claude Shannon (1937, published the definitive mathematical proof mapping Boolean algebra to electrical relays), George Stibitz (1937, constructed the first electromechanical binary adder), and John Vincent Atanasoff (1937, adopted binary to keep the vacuum-tube count of electronic circuits physically viable). + Also of interest, Turing restricted the figures printed by his a-machine to 0 and 1, so that the sequences it computes are binary. George Boole's work (1847, 1854) was well established by then, so from a theoretical standpoint, it was a sensible simplification. However, utilizing binary within the context of a machine description effectively bridged the gap to the more practically minded engineers of the time. Alan Turing's paper arrived at the same time that switched telephone networks had reached a scale that made them difficult to maintain without systematic approaches. These networks were built upon electromechanical relays, which were decisively binary devices. At least seven men in addition to Alan Turing appear to have independently contemplated the intersection of Boolean algebra, logic, and physical computing: Victor Shestakov (1935, proposed mapping Boolean algebra to electromechanical relay circuits), Konrad Zuse (1936, adopted base 2 architecture to bypass the physical complexity of decimal mechanical gears), Akira Nakashima (1936, published the mathematical equivalence of Boolean algebra and two-terminal switching networks), Louis Couffignal (1936, argued calculating machines must shift to binary linkages to reduce physical friction), Claude Shannon (1937, published the definitive mathematical proof mapping Boolean algebra to electrical relays), George Stibitz (1937, constructed the first electromechanical binary adder), and John Vincent Atanasoff (1937, adopted binary to keep the vacuum-tube count of electronic circuits physically viable).

- To complete the Turing Machine story, then, we will transform the Turing Machine into a modern architecture in a stepwise fashion, while ensuring that at each step the modifications are inconsequential to computation theoretic existence proofs and complexity class results. We do run into some problems, so the architecture we derive will be a little different from those we currently build. Though the point here is that we could build it. + Before asking whether a given architecture measures up to the Turing Machine, we need to say what measuring up would consist of. Like a Turing Machine, a computer architecture is an abstraction. The box sitting on a person's desk is a realization of some computer architecture. To say a Turing Machine does something is to say that the Turing Machine was analyzed and the result of the analysis is that 'something'. A computer architecture can also be analyzed. A computer architecture is said to be Turing Complete when, through analysis, it is determined that it can do anything that a Turing Machine can do. The practical implication for a realization of such an architecture is that running a program will fault only because a) the program logic told it to, b) the program has a flaw, or c) there is a mathematical fact standing in the way of execution. A shortage of a physical resource is not a fourth reason, provided the architecture can pause a program until a 'more memory' request is fulfilled, because that shortage is a limit of the realization and not of the architecture. However, if the architecture itself stipulates a bound that a program can reach, such as a fixed address width or a fixed integer width, then every realization of it must fail on some program that a Turing Machine would carry to completion, and the architecture is not Turing Complete.

- The infinite tape is not as large a hurdle as it might seem at first. For computational problems, the Turing Machine halts in a finite number of steps. Because the Turing Machine is limited to stepping the read/write head over one cell per machine execution step, only a finite amount of tape is ever used. But for a given computation, how much tape is that? Resolving this by assuming more tape is simply attached when needed is analogous to cheating in a 'guess the bigger number' game by declaring, "My number is always one bigger than the given number, so I will tell you my guess after you state your number." Some mathematicians suggest that what is meant by infinity is precisely a rule of this sort. For engineers building physical hardware, however, to state that a resource starts finite and expands incrementally over time is a very different proposition from being asked to install infinite memory on a machine in the first place. + With that criterion in hand, consider the infinite tape, which is not as large a hurdle as it might seem at first. For computational problems, the Turing Machine halts in a finite number of steps. Because the Turing Machine is limited to stepping the read/write head over one cell per machine execution step, only a finite amount of tape is ever used. But for a given computation, how much tape is that? Resolving this by assuming more tape is simply attached when needed is analogous to cheating in a 'guess the bigger number' game by declaring, "My number is always one bigger than the given number, so I will tell you my guess after you state your number." Some mathematicians suggest that what is meant by infinity is precisely a rule of this sort. For engineers building physical hardware, however, to state that a resource starts finite and expands incrementally over time is a very different proposition from being asked to install infinite memory on a machine in the first place.

- In 1967, Marvin Minsky addressed this very topic, saying: "We need not think of the machine's tape as infinite. We imagine instead that the machine begins with a finite tape, but that, whenever an end is encountered, another unit of tape is attached." Marvin L. Minsky, Computation: Finite and Infinite Machines (Englewood Cliffs: Prentice-Hall, 1967), 167. In 1967, this was a perfectly natural thing to suggest, as computers utilized magnetic tape memory on manually mounted reels, and it was entirely possible for a computation to stop and request a new reel of tape to be mounted. Contemporary computer architectures do, in fact, achieve a similar effect through virtual memory. When physical RAM is depleted, the operating system pauses the active process and autonomously provisions apparent capacity by swapping memory pages out to auxiliary storage. However, this illusion of infinite tape remains bound by the physical limits of the secondary storage and the boundaries of the processor's address space. Once the available swap space is exhausted or the address space is saturated, the operating system abruptly terminates the process. + In 1967, Marvin Minsky addressed this very topic, saying: "We need not think of the machine's tape as infinite. We imagine instead that the machine begins with a finite tape, but that, whenever an end is encountered, another unit of tape is attached." Marvin L. Minsky, Computation: Finite and Infinite Machines (Englewood Cliffs: Prentice-Hall, 1967), 167. In 1967, this was a perfectly natural thing to suggest, as computers utilized magnetic tape memory on manually mounted reels, and it was entirely possible for a computation to stop and request a new reel of tape to be mounted. Contemporary computer architectures do, in fact, achieve a similar effect through virtual memory. When physical RAM is depleted, the operating system pauses the active process and autonomously provisions apparent capacity by swapping memory pages out to auxiliary storage. So the graceful expansion Minsky described is already in place, and it works, right up until a stipulated bound is reached. Once the available swap space is exhausted or the address space is saturated, the operating system abruptly terminates the process.

- Like a Turing Machine, a computer architecture is an abstraction. The box sitting on a person's desk is a realization of some computer architecture. To say a Turing Machine does something is to say that the Turing Machine was analyzed and the result of the analysis is that 'something'. A computer architecture can also be analyzed. A computer architecture is said to be Turing Complete when, through analysis, it is determined that it can do anything that a Turing Machine can do. The practical implication for a realization of such an architecture is that running a program will fault only because a) the program logic told it to, b) the program has a flaw, or c) there is a mathematical fact standing in the way of execution. It is tolerable to call a computer architecture Turing Complete if it has the built-in ability to pause a program until a 'more memory' request is fulfilled, as that is a limit of the realization rather than of the architecture. However, if the architecture itself stipulates a bound that a program can reach, such as a fixed address width or a fixed integer width, then every realization of it must fail on some program that a Turing Machine would carry to completion, and the architecture is not Turing Complete. + Notice where the failure actually sits. Expanding the memory available to a process is handled, and handled well. What is missing is the ability to continue rather than to terminate, and supplying that is software's work, much as Python supplies integers of arbitrary range over a machine whose integers are of fixed width. The difference is one of motive. Python's arbitrary range integers are a convenience offered to the programmer, whereas here the software would be compensating for a bound the architecture imposed. This is the shape of the whole problem in miniature: an architecture is made to look complete by software written to work around it, rather than being complete by virtue of its own instructions.

- +

+ The Turing Machine does not escape criticism either, and its defect is the more serious of the two. Its head must read and react at every step, and the only place a controller has to hold what it has seen is a branch in its own control path. So the Turing Machine uses its controller as memory. When we come to analyze a machine that reverses a string, we will find the consequence: the number of states and arcs required grows exponentially against the width of a machine word, and for a word of any practical size the controller cannot be built at all. A machine room operator can mount another reel of tape. Nobody can mount a larger controller, because the controller is finite by definition. The tape's limit was handed outside the machine and dealt with there. The controller's limit is sealed inside the model, where nothing can reach it. +

+

+ To complete the Turing Machine story, then, we will transform the Turing Machine into a modern architecture in a stepwise fashion, while ensuring that at each step the modifications are inconsequential to computation theoretic existence proofs and complexity class results. The transformation does not run in one direction only. On the Turing Machine side, the controller has to stop being used as memory. On the modern side, the fixed widths an architecture stipulates, of an address and of an integer, are what have to give way. The two meet in the middle, and the machine we arrive at is less strange than that might suggest. It separates the control path from the data path, it holds an instruction table, and it has a small register file. It looks modern. The differences from what we currently build are real but few, and the point of the exercise is that we could build it. +

+ The computer design abstraction stack @@ -274,31 +283,31 @@
    -
  1. mathematical logic
    2. -
  2. computation theory
    3. -
  3. architecture
    4. -
  4. organization
    5. -
  5. implementation
    6. -
  6. realization
    7. +
  7. mathematical logic
    8. +
  8. computation theory
    9. +
  9. architecture
    10. +
  10. organization
    11. +
  11. implementation
    12. +
  12. realization
List . The six levels of the computer design abstraction stack

- Mathematical logic underpins the computation theory layer. Computation theory speaks of the time and space complexity of algorithms and the existence of solutions to decider problems, which in turn guides the goals of the architecture and organization layers. + Mathematical logic underpins the computation theory level. Computation theory speaks of the time and space complexity of algorithms and the existence of solutions to decider problems, which in turn guides the goals of the architecture and organization levels.

- An architecture provides programmers with information that is valuable when designing the logic of programs. This includes programmers across the entire software stack, such as firmware engineers, driver developers, systems programmers, compiler writers, and application developers. Although applications programs might only be exposed to the virtual architecture presented by various standards, various libraries, and the programming language used. In addition to specifying the instructions (instruction set architecture), architecture includes describing the memory and hardware virtualization features, specifying the behavior of the interrupt subsystem, the method of doing I/O, DMA, the special registers and their effects, any architectural busses, and the standards to be followed for each if any. More recently, this also includes specifying how programs can make use of secure areas. The architecture is specified by an architect. + An architecture provides programmers with information that is valuable when designing the logic of programs. This includes programmers across the entire software stack, such as firmware engineers, driver developers, systems programmers, compiler writers, and application developers, although application programs might only be exposed to the virtual architecture presented by the standards, the libraries, and the programming language in use. Beyond specifying the instructions, which is the instruction set architecture, an architecture describes the memory and hardware virtualization features, the behavior of the interrupt subsystem, the method of doing I/O, DMA, the special registers and their effects, any architectural buses, and the standards to be followed for each, if any. More recently, this also includes specifying how programs can make use of secure areas. The architecture is specified by an architect.

- The classic text by Hamacher, Vranesic, and Zaky carefully defines the organizational level as sitting between architecture and implementation V. Carl Hamacher, Zvonko G. Vranesic, and Safwat G. Zaky, Computer Organization, 5th ed. (New York: McGraw Hill, 2002).. - Organization is the register transfer level description of the machine, which includes internal buses, external buses and the state machines that implement the protocols used, control units, interrupt structures, and ALU layout. Crucially, it is at this level that decisions regarding instruction level parallelism are made, such as whether the processor will employ a scalar, superscalar, or VLIW design, the depth of its execution pipelines, the use of out-of-order execution, branch prediction strategies, and the specific hierarchy of hardware caches. It dictates the logical arrangement of hardware and the procedures that force the data to flow to satisfy the architectural constraints. Organization is sometimes called micro architecture, and it is made by a design architect. + The classic text by Hamacher, Vranesic, and Zaky carefully defines the organizational level as sitting between architecture and implementation V. Carl Hamacher, Zvonko G. Vranesic, and Safwat G. Zaky, Computer Organization, 5th ed. (New York: McGraw-Hill, 2002).. + Organization is the register-transfer level description of the machine, which includes internal buses, external buses and the state machines that implement the protocols used, control units, interrupt structures, and ALU layout. Crucially, it is at this level that decisions regarding instruction-level parallelism are made, such as whether the processor will employ a scalar, superscalar, or VLIW design, the depth of its execution pipelines, the use of out-of-order execution, branch prediction strategies, and the specific hierarchy of hardware caches. It dictates the logical arrangement of hardware and the procedures that force the data to flow to satisfy the architectural constraints. Organization is sometimes called microarchitecture, and it is made by a design architect.

- It is not a requirement of a computer organization, nor an architecture, that it be capable of physical realization. The abstract Turing Machine organization developed in chapter serves as an example. Instead, an abstract organization can serve other purposes, in this case as a stepping stone to another organization that can be realized. + It is not a requirement of a computer organization, nor of an architecture, that it be capable of physical realization. The abstract Turing Machine organization developed in chapter serves as an example. Instead, an abstract organization can serve other purposes, in this case as a stepping stone to another organization that can be realized.

@@ -306,11 +315,11 @@

- A realization is a physical box full of plastic, metal, fiberglass, and silicon, along with a smattering of exotic materials. A realization is made by manufacturing engineers, technicians, and product line workers, with the assistance of some of the most sophisticated machines ever built by humankind. + A realization is a physical box full of plastic, metal, fiberglass, and silicon, along with a smattering of exotic materials. A realization is made by manufacturing engineers, technicians, and product line workers, with the assistance of some of the most sophisticated machines ever built by humankind.

- If a computer manufacturer keeps the architecture as a constant, all other levels can change, and a customer will be able to run the same software. The same organization can be used with different implementations. Minor changes in manufacturing process can sometimes be used with an older implementation, for example a simple transistor shrink. + If a computer manufacturer keeps the architecture constant, all other levels can change, and a customer will be able to run the same software. The same organization can be used with different implementations. Minor changes in the manufacturing process can sometimes be used with an older implementation, for example a simple transistor shrink.

@@ -319,23 +328,23 @@ The levels are not independent

- The layers are merely idealizations. In both practice and theory it is not possible to completely disentangle them. On a new machine of the same architecture, it is common that some software will require updates to run, and almost certainly specific operating system support will be required. + The levels are merely idealizations. In both practice and theory it is not possible to completely disentangle them. On a new machine of the same architecture, it is common that some software will require updates to run, and almost certainly specific operating system support will be required.

- An architect almost always has a reference organization in mind, and design architects work with design engineers to know what is practical, and design engineers work with manufacturing engineers to know what can be built. + An architect almost always has a reference organization in mind. Design architects work with design engineers to know what is practical, and design engineers work with manufacturing engineers to know what can be built.

- The common understanding of the word 'architecture' is what Hamacher and Zaky call an 'organization'. For example, even the most experienced of architects will say things like a microprocessor has a "superscalar architecture", though whether a processor is a scalar, superscalar, is clearly a question of computer organization. + The common understanding of the word 'architecture' is what Hamacher, Vranesic, and Zaky call an 'organization'. For example, even the most experienced of architects will say things like a microprocessor has a "superscalar architecture", though whether a processor is scalar or superscalar is clearly a question of computer organization.

- In fact, architecture instructs organization. When an architect designs an instruction set that has load instructions, it implies that there will be an instruction fetch, and thus an instruction bus. Furthermore the load data has to come from somewhere, so there will be data fetch and a data bus. Could both be the same bus? If not, then we have a "Harvard Architecture". The fact is, almost no one involved in computer design completely divorces architecture from organization. + In fact, architecture instructs organization. The fact there is an instruction set already means that there will be instruction fetch, and thus an instruction bus. Furthermore the load data has to come from somewhere, so there will be data fetch and data bus. Could both be the same bus? If not, then we have a "Harvard Architecture". The fact is, almost no one involved in computer design completely divorces architecture from organization.

- This cascades down the stack, as organization instructs implementation, etc. For example, if the architecture has an instruction that names one of N registers as an operand, then the organization has a register file that data flows to and from, and busses to carry that data, the design will specify a register file and layout the busses, and the manufacturing people will build them. + This cascades down the stack, as organization instructs implementation, and implementation instructs realization. For example, if the architecture has an instruction that names one of N registers as an operand, then the organization has a register file that data flows to and from, along with buses to carry that data. The implementation then specifies that register file and lays out those buses, and the manufacturing people build them.

@@ -344,12 +353,28 @@ Where the Turing Machine fits in

- The Turing Machine is a computation theory object that is suggestive of a simple architecture, and a computer organization. A person who has had to do homework problems centered on Turing Machines will have tracked the flow of data through the machine, i.e. worked at the register transfer level. However, a little work is needed to complete the architecture analog. The fundamentals are present, the read/write head, the tape, the procedure for using the tape, but other components are missing. The manipulation of symbols remains ungrounded. The tape is not well defined. The use of emptiness is non architectural like. The tape transport is not articulated, though it is implied. The read buffer that is required, so the programmed controller can do a write without clobbering the read data needed for the next transition, is not identified as a component. As we proceed, we will likely discover other missing components. + The Turing Machine is a computation theory object that is suggestive of a simple architecture, and a computer organization. A person who has had to do homework problems centered on Turing Machines will have tracked the flow of data through the machine, i.e. worked at the register-transfer level. However, a little work is needed to complete the architecture analog. The fundamentals are present: the read/write head, the tape, and the procedure for using the tape. Other things are missing, or are left unspecified: +

+ + + +
    +
  1. The manipulation of symbols remains ungrounded.
    2. +
  2. The tape is not well defined.
    3. +
  3. The use of emptiness is not architectural.
    4. +
  4. The tape transport is not articulated, though it is implied.
    5. +
  5. The read buffer is not identified as a component. One is required so that the programmed controller can do a write without clobbering the read data needed for the next transition.
    6. +
+ List . What the Turing Machine leaves out of its architecture + + +

+ As we proceed, we will likely discover other missing components.

- + Computation theoretic consequentiality diff --git a/document/todo.txt b/document/todo.txt index 2554986..4674623 100644 --- a/document/todo.txt +++ b/document/todo.txt @@ -1,3 +1,7 @@ +reverse the numbering of list items on the abstraction stack + + + 2026-03-26 03:41:44 when making a skeleton from Harmony, set skeleton docs and other files not to be edited to read only