From: Thomas Walker Lynch Date: Sat, 11 Jul 2026 12:24:53 +0000 (+0000) Subject: . X-Git-Url: https://git.reasoningtechnology.com/?a=commitdiff_plain;h=9b666bc954ab01c9572cf34b0ff2e131ec2e1eab;p=TM-2026 . --- diff --git a/document/TM-2026.html b/document/TM-2026.html index b40f113..a79900d 100644 --- a/document/TM-2026.html +++ b/document/TM-2026.html @@ -19,13 +19,7 @@ - Preface - -

- This is the second edition of "Tom's Turing Complete Computer Architecture" first volume. It now has its own title, its down introduction, and improved descriptions. -

- - The search for a formal foundation for mathematics led to the Turing Machine + The search that led to the Turing Machine

In 1893 Gottlob Frege published an axiomatic construction of mathematics from set theory. Frege's grand objective was something he called Logicism, the philosophical thesis that all of mathematics can be derived entirely from pure logic. To bridge set theory and logic, Frege defined sets using a method known as set comprehension. Under this approach, a mathematician states a logical rule or property, and any object satisfying that logical statement automatically becomes a member of the set. Because the membership of a set is determined entirely by logical rules, the resulting sets, and the mathematics built upon them, are derived directly from logic. To implement this, his specific machinery relied upon unrestricted set comprehension, formalized as Basic Law V Gottlob Frege, Grundgesetze der Arithmetik, Vol. 2 (Jena: Hermann Pohle, 1903), Appendix (Nachwort), 253.. @@ -842,7 +836,7 @@

- Computational Naturalism + Or, does it go the other way? Computational Naturalism

Given that we have an architectural definition for a Turing Machine, and will modify this in a later chapter to define a realizable Turing Complete architecture, it is possible to invert the foundation of mathematics. Instead of mathematics preceding computation, we posit that given a Turing Machine exists, all of mathematics is an interpretation of what can be done with it. @@ -1768,11 +1762,11 @@

- In contrast, a function extension version of the Natural Number sequence generator can be run. The function extension machine would be given as input a prior function extension result, or an initial empty tape. It would then modify the tape to contain the next Natural Number, as per the sequence that the Natural Number machine would write, if it could be run. This is accomplished through two subroutine calls: find_empty and increment. The find_empty machine checks the symbol under the head. Upon finding it is not the empty symbol, the machine steps right and checks the next cell, repeating until it finds the empty symbol, whereupon it halts. The increment machine writes an 's' onto the tape and halts. Recurrent calls to the Natural Number extender then produce a sequence of result tapes: [], [s], [ss], …. Similarly, a programmer can write a machine called decrement, though the programmer must note that decrementing can return the left-of-leftmost symbol. + In contrast, a function extension version of the Natural Number sequence generator can be run. The function extension machine would be given as input a prior function extension result, or an initial empty tape. It would then modify the tape to contain the next Natural Number, as per the sequence that the Natural Number machine would write, if it could be run. This is accomplished through two subroutine calls: find_empty and increment. The find_empty machine checks the symbol under the head. Upon finding it is not the empty symbol, the machine steps right and checks the next cell, repeating until it finds the empty symbol, whereupon it halts. The increment machine then writes an 's' onto the tape and halts. Recurrent calls to the Natural Number extender then produce a sequence of result tapes: [], [s], [ss], …. Similarly, a programmer can write a machine called decrement, though the programmer must note that decrementing can return the left-of-leftmost symbol.

- To say that Natural Number A is smaller than Natural Number B is to say that A would occur on the Natural Number Machine tape to the left of B, if the machine were run. Similarly, if B were said to be greater than A, that means B would occur further to the right. It is a simple matter for a programmer to write decider machines for this. The decider is given two input tapes for the two numbers to compare, and it keeps a head on each. It then steps forward until neither head has an 's' under it. If empty symbols are found simultaneously on both tapes, then the two numbers are equal; otherwise, the number with an empty symbol under the head is the lesser number. + To say that Natural Number A is smaller than Natural Number B is to say that A would occur on the Natural Number Machine tape to the left of B, if the machine were run. Conversely, if B were said to be greater than A, that would mean B occurs further to the right. It is a simple matter for a programmer to write a decider machine for this. The decider is given two input tapes for the two numbers to compare, and it keeps a head on each. It then steps forward until neither head has an 's' under it. If empty symbols are found simultaneously on both tapes, then the two numbers are equal; otherwise, the number with an empty symbol under the head is the lesser number.

@@ -1791,7 +1785,7 @@

Binary Arabic representation address

- Would it be computationally consequential if binary Arabic representation were used instead of unary for addresses? For unary representation each increment was constant time, but with Arabic addresses, increments would have a carry. Would the carry cause stepping to the right, combined with calling the address machine, to become slower and slower as the machine steps further and further? + Would it be computationally consequential if binary Arabic representation were used instead of unary for addresses? For unary representation each increment was constant time, but with Arabic addresses, increments have a carry. Would the carry cause stepping to the right, combined with calling the address machine, to become slower and slower as the machine steps further and further?

@@ -1813,7 +1807,7 @@ goto increment -

Here TTU is the tape transport unit. It has the number to be incremented mounted on it. This number is either zero, which would mean the leftmost cell is empty, or it is of the same form as a result from a Natural Number extension machine. A TTU.read places a copy of the symbol instance found in the cell under the head into the read buffer automatically when the machine enters a new state on the programmed controller, so it is not counted as a step. In contrast, the other actions are associated with the state of the programmed controller, so when the machine arrives at a write, step, or halt node, it means that the machine has taken a step. Sequential instructions mean unconditional next state choices, whereas an 'if' signals a conditional next state choice. The condition is a logical proposition based on the read symbol.

+

Here TTU is the tape transport unit. It has the number to be incremented mounted on it. This number is either zero, which would mean the leftmost cell is empty, or it is of the same form as a result from a Natural Number extension machine. A TTU.read places a copy of the symbol instance found in the cell under the head into the read buffer automatically when the machine enters a new state on the programmed controller, so it is not counted as a step. In contrast, the other actions are associated with the state of the programmed controller, so when the machine arrives at a write, step, or halt node, it means that the programmed state controller has taken a step. Sequential instructions mean unconditional next state choices, whereas an if signals a conditional next state choice. The if condition is a logical proposition based on the read symbol.

The loop form here is worth taking note of, as it will come up again. The controller executes a first action, that of a read, followed by a middle break test, and then the recursive form actions.

@@ -1870,7 +1864,7 @@

- The number of programmable controller state transition steps required to reach a maximum 1 bit, 2 bit, and 3 bit count are: [2, 8, 22]. Note these are cumulative. The three bit maximum count cost includes that of the two bit maximum count cost, etc. Each cycle becomes exponentially longer, so a better measure is the average cost to reach each maximum count: [2/1, 8/3, 22/7] = [2, 2.667, 3.143] + The number of programmable controller state transition steps required to reach a maximum 1 bit, 2 bit, and 3 bit count are: [2, 8, 22]. Note these are cumulative. The three bit maximum count cost includes that of the two bit maximum count cost, etc. Each maximum count is exponentially larger, so a better measure is the average cost per step: [2/1, 8/3, 22/7] = [2, 2.667, 3.143]

@@ -1880,7 +1874,7 @@ \sum_{k=1}^{n} 2k \cdot 2^{n-k} = 2^{n+2} - 2n - 4

- To find the average cost per increment across this entire progression, a person divides the total number of tape actions by the total number of increments, which is 2^n - 1: + To find the average cost per increment to reach each maximum, an analyst divides by the total number of increments, which is 2^n - 1:

\text{Average Cost} = \frac{2^{n+2} - 2n - 4}{2^n - 1} = \frac{4(2^n-1)+4-2n-4}{2^n-1} @@ -1892,7 +1886,7 @@ \text{Average Cost} = 4 - \frac{2n}{2^n - 1}

- As the address space grows, the bit width n increases. The polynomial term 2n is outpaced by the exponential denominator 2^n - 1, causing the fractional terms to converge to zero. The average work done by the machine head converges to exactly four tape actions per increment. This is a surprising counter intuitive result. + As the address space grows, the bit width n increases. The polynomial term 2n is outpaced by the exponential denominator 2^n - 1, causing the fractional terms to converge to zero. The average work done by the machine head converges to exactly four tape actions per increment. Constant time, same as for unary. This is a surprising counter intuitive result.

@@ -1900,21 +1894,92 @@

- The model used here to define an address is analogous to keeping a pointer into memory, and then using that pointer value as the address. Each step then increments or decrements the pointer. A pointer will be one word in width. In small geometries where computation element delay dominates, two operand adders of fixed width tend towards logarithmic complexity evaluation times against bit width, which makes it practical to perform word width addition operations atomically in one machine cycle. In fact, for some machines, the adder time likely sets the clock period. Consequently, a programmer cannot arrange a program to expose increment time aliasing to the real world. It remains a theoretical observation. -

+ The model used here to define an address is analogous to keeping a pointer into memory. Each step then increments or decrements the pointer. Conventional processors do the converse of the Turing Machine model presented here. For a processor, the address is the base object, while the head location is what is implied by the address. On processors, pointers will be one word in width. In small geometries where computation element delay dominates, two operand adders tend towards logarithmic complexity evaluation times against bit width, and the constants are small. Because of this, it is practical to perform word width addition operations atomically in one machine cycle. In fact, for some machines, the adder time likely sets the clock period. Such an adder will be used for address arithmetic; consequently, a programmer cannot arrange a program to expose aliasing in the address increment time to the real world. It remains a theoretical observation. +

- On real machines, there is no head sitting on top of system memory that gets pushed back and forth. Instead, pointers are sent on a trip through a virtual memory system, and then through an address decoder so as to locate a memory cell. Address decoders resemble carry chains, and they also have logarithmic physical evaluation times as word width grows. Hence, a fixed width word holding an address gets decoded in approximately constant time measured in clock ticks. Address decoding will get spread across the memory hierarchy, so the full answer is that decode time is a complex question. + An address is sent on a trip through the memory hierarchy to locate the corresponding memory cell, ultimately landing on a memory decoder over a physically implemented memory. Address decoders resemble carry chains, and they also have logarithmic physical evaluation times as word width grows. Hence, a fixed width word holding an address gets decoded in approximately constant time measured in clock ticks. However, the trip through the memory hierarchy is not negligible.

- Although addresses are decoded, and thus locality plays no role, each decode being independent, locality of reference does matter on real machines. In interleaved memory, accesses to one location will also pull down neighbors. Prefetch algorithms will assume that the contents of neighbors are also needed, and cache lines organize memory in blocks, so the contents of each cell again comes along with its neighbors.

+ Direct address decoding operates independently of reference locality. This fundamentally differs from a Turing Machine head, which incurs a linear computation cost strictly proportional to the distance traveled. However, direct decoding is only the base of the memory hierarchy. +

- For very large programs, when the virtual memory system becomes salient, the memory architecture begins to look more like that of a Turing Machine. The translation lookaside buffer provides stateful location context, and the neighbor relationship between pages is sometimes taken into account for performance reasons. However, once a program starts performing at virtual memory page fetch times instead of local system memory access times, this is known as page thrashing and when it occurs programs become very slow. It is also possible for small programs to use address access patterns that lead to page thrashing. This is another form of address aliasing. + It is possible that decoding occurs separately in a row and column manner, and changes in the lower bits of the address walk a buffered row that was found in a prior access. This makes access to neighbor cell addresses within a row a low expense operation. However, this depends on memory implementation. One layer up, it is common for memory to be made of interleaved banks, so neighbor access that remains in the leaf collection is generally faster than that which requires fetching a new leaf set.

-

So in summary then, using Arabic representation for addressing is not statistically computationally consequential. However, the idea of using a decoded address in place of a tape head raises issues. It would not be an exaggeration to say that the goal of modern computer architecture is to ensure that within the locality of an execution context, address decoding does not become computationally consequential. +

+ There will, however, be a more dominant effect. For programs with islands of locality of reference, instructions or data will be discovered in a cache. Caches bring in two types of locality: that of access within a cache line, which makes neighbor walks that stay in the line fast, and cache line prefetch and replacement policies, which assume for any given fetch address that a subsequent fetch will be nearby in memory. These effects lead to similar memory behavior as a head on tape, where a single step to a neighbor is faster than taking many steps to find memory further away, but there is not a one to one correspondence. +

+ +

+ Memory accesses into the processor local memory file are even faster. If locality of reference, even in an abstract sense, remains tight, then memory cells can be accessed in a minimal amount of time. +

+ +

+ For programs with large memory footprints and sparse memory access patterns, the dominant factor becomes the virtual memory subsystem. Here locality again comes into play as the translation lookaside buffer provides stateful location context, and the neighbor relationship between pages is sometimes taken into account for performance reasons. However, once a program starts performing at virtual memory page fetch times instead of cache access, or even local system memory access times, the program is said to be page thrashing, and it becomes very slow. It is also possible for small programs to have memory access patterns that alias against the memory subsystem's heuristics and lead to page thrashing. For a typical system, a programmer can write a small program to demonstrate this. +

+ +

+ In summary then, using Arabic representation for addressing is not statistically computationally consequential. On the other hand, the idea of using a decoded address in place of a tape head raises computational model issues. So much so that it would not be an exaggeration to say that the very goal of modern computer architecture is to ensure that, within an execution context, address decode does not become computationally consequential. Architects employ a memory hierarchy to do this, and that memory hierarchy rewards locality of reference, but access times are not linear in time relative to distance from prior access. Instead they are clumped into a constant time access abstract concept of locality groups where the time to access each group grows exponentially with the distance from the processor. +

+ +

+ Let us put this into perspective. Suppose in ancient Roman times that a clock tick for a computer was scaled to be one day long, so that the action involves sending letters rather than variable values. Under this scale, a single nanosecond of real world execution time equates to three days. Suppose a program initiated a read request for a location in memory on the Ides of March, the date when Caesar was assassinated on -0043-03-15. Surely that was a big day for posting letters. The following table provides the historical date that the variable value would finally be loaded into the processor, depending on the memory tier being accessed: +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Memory TierReal World LatencyScaled DelayScaled Arrival EraHistorical Context
L1 Cache Hit1 ns3 days-0043-03-18Three days after the assassination.
DRAM (Main memory)100 ns300 days-0042-01-09Nearly a year later, during the Liberators' civil war.
NVMe SSD Page Swap15 µs45,000 days0080123 years later, exactly as the Colosseum is completed in Rome.
SATA SSD Page Swap100 µs300,000 days0778821 years later, during the reign of Charlemagne and the Frankish Empire.
Magnetic HDD Page Swap10 ms30,000,000 days82092Tens of thousands of years in the future, long after current human civilizations are dust.
+ +

+ In computer architecture, there is a rule that the slower the access time, the larger the amount of data that should be transferred. The idea is then that of access time per bit, instead of access time per fetch. A common example given is that of a station wagon of hard drives driven across the country. Andrew S. Tanenbaum famously coined this concept in his 1981 textbook Computer Networks, stating: "Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway." This physical data transfer method is universally referred to as a sneakernet. Ten terabits, at three days, equates to 26 nanoseconds per bit. Not bad, if a user can wait three days to continue the program, but aliasing between data sets would make a program very slow. The Turing Machine model does not capture the effects of clumped data accesses.

@@ -2025,20 +2090,34 @@

A given finite area that is not sitting on the end of the tape will have a left neighbor area and right neighbor area that are not in the area. If the given area is sitting on the end of the tape there will be no left neighbor area. If the area is open on the right, there will be no right neighbor area.

-

Now imagine we define a Turing Machine such that when it steps right, it steps to the right neighbor area, and when it steps left, if there is a left area, it steps to that, otherwise it returns a left of leftmost symbol. The read instruction reads all of the cells in the area, and similarly the write instruction writes them all. Then for this machine the areas are virtual tape cells. +

Now imagine we define a Turing Machine such that when it steps right, it steps to the right neighbor area, and when it steps left, if there is a left area, it steps to that, otherwise it returns a left-of-leftmost symbol. The read instruction reads all of the cells in the area, and similarly the write instruction writes them all. Then for this machine the areas are virtual tape cells.

An example of this from conventional computing is where memory is accessed in units of bytes, but a program works with units of words.

Virtual tape

-

As established, the Turing Machine head transport returns a distinct left of leftmost symbol when stepping left off the end of the tape. The programmed controller, the statement machine, then uses an arc defined specifically for this symbol. It is a distinct symbol that cannot be written to the tape. +

As established, the Turing Machine head transport returns a distinct left-of-leftmost symbol when stepping left off the end of the tape. The programmed controller, the statement machine, then uses an arc defined specifically for this symbol. It is a distinct symbol that cannot be written to the tape. +

+ +

Various portions of this text have mentioned calling a Turing Machine as a subroutine. This is described as unmounting the tape from the calling machine, mounting it on the subroutine machine, running the subroutine machine until it halts, then unmounting the tape from the subroutine machine, and mounting it back on the calling machine. However, this would be a manual process. To automate the process, we use a Universal Turing Machine. It has the definition for the calling machine and the subroutine machine on its tape. We can add an additional tape to the machine to hold the stack. Then a subroutine call happens much as it does in conventional software. +

+ +

A gasket machine is actually a set of subroutines with one corresponding to each of the tape transport commands. +

+ +

+ There is a small problem, in that the left-of-leftmost symbol was stipulated as being separate from the alphabet, and only alphabet symbols and the empty symbol can be written to the tape. There are a couple of possible solutions. One solution is to add a symbol to the alphabet that is also taken to mean 'left of leftmost', say gasket-left-of-leftmost. Then machines that make use of gaskets can recognize this new symbol for its meaning. Another alternative, that will be used for the remainder of our discussion, is to define a separate control alphabet of symbols that are distinct from data alphabet symbols, but can also be written and read from the tape. We will add to this alphabet, empty, left-of-leftmost, and a new control symbol used by gaskets,right-of-rightmost.

Now suppose defining a Turing Machine that initially has the head on the leftmost cell of a specific area. When step left is called, the tape transport immediately returns the left of leftmost symbol, independent of whether the area is actually at the physical left end of the tape or not. +

+ +

+Now suppose defining a Turing Machine that initially has the head on the leftmost cell of a specific area. When step left is called, the tape transport immediately returns the left of leftmost symbol, independent of whether the area is actually at the physical left end of the tape or not.

-

Suppose further that this area is finite. If the machine attempts to step right from the rightmost cell of this finite area, the tape transport returns, in analogy, a right of rightmost symbol. Such a machine defines a virtual tape over an area. +

Suppose further that this area is finite. If the machine attempts to step right from the rightmost cell of this finite area, the tape transport returns, in analogy, right-from-rightmost symbol. Such a machine defines a virtual tape over an area.

Partitions on a finite virtual tape