From: Thomas Walker Lynch Date: Fri, 31 Jul 2026 02:53:33 +0000 (+0000) Subject: start of Hindu-Arabic chapter revision X-Git-Url: https://git.reasoningtechnology.com/%27%20%20%20full_path%20%20%20%27?a=commitdiff_plain;h=6444839ed12945285c73e0b1c5c409df006427bf;p=TM-2026 start of Hindu-Arabic chapter revision --- diff --git a/document/book/Arabic number fig 1.png b/document/book/Arabic number fig 1.png new file mode 100644 index 0000000..ea8cc48 Binary files /dev/null and b/document/book/Arabic number fig 1.png differ diff --git a/document/book/Arabic number fig 2.png b/document/book/Arabic number fig 2.png new file mode 100644 index 0000000..17fa599 Binary files /dev/null and b/document/book/Arabic number fig 2.png differ diff --git a/document/book/Arabic number fig 3.png b/document/book/Arabic number fig 3.png new file mode 100644 index 0000000..1f1935a Binary files /dev/null and b/document/book/Arabic number fig 3.png differ diff --git a/document/book/Arabic number fig 4.png b/document/book/Arabic number fig 4.png new file mode 100644 index 0000000..c2004e6 Binary files /dev/null and b/document/book/Arabic number fig 4.png differ diff --git a/document/book/TM-2026.html b/document/book/TM-2026.html index da20e69..391ffc9 100644 --- a/document/book/TM-2026.html +++ b/document/book/TM-2026.html @@ -713,12 +713,12 @@ \begin{aligned} \mathit{AF} = \{& \\ - & \mathtt{no\_op} \\ - , & \mathtt{left} \\ - , & \mathtt{right} \\ - , & \mathtt{read}(\mathtt{d} \mid \mathtt{s}) \\ - , & \mathtt{status} \\ - , & \mathtt{write}(\mathtt{d} \mid \mathtt{s} \mid \mathtt{σ}, [σ]) \\ + & \mathtt{no\_op} \\ + ,& \mathtt{left} \\ + ,& \mathtt{right} \\ + ,& \mathtt{read}(\mathtt{d} \mid \mathtt{s}) \\ + ,& \mathtt{status} \\ + ,& \mathtt{write}(\mathtt{d} \mid \mathtt{s} \mid \mathtt{σ}, [σ]) \\ \} \end{aligned} @@ -1272,7 +1272,7 @@

The prior chapter on the computation theoretic TTCA machine serves as the architectural template, with only a few modifications. The architecture requires explicit data rather than accepting meta-symbols like 'unspecified' as presumed initial values 'by definition'. Actual values are transacted. Now that data and control have been separated, the controller is practical to implement, and even more so because it was defined in terms of tables that can be built in hardware. As the read status instruction returns the cell type, it will in its current form be able to return 'rightmost', so the right end of the tape can be detected. In order to extend the tape, the machine will stop and ask the operator to mount a new reel. This could be signaled when the user attempts to step right of rightmost, either by a panel light that illuminates upon the machine finding a rightmost status, or by the program printing a message on the console teletype. As this is a constant-time operation, it is computation theoretic inconsequential.

-

Because a Turing Machine can only reach another cell further out on the tape by stepping to it, space complexity and time complexity are related. A program that runs for ten steps can consume at most ten cells of tape. However, if that program merely bounces between two cells, it will require less space, precisely two cells. As another example, a program that counts the number of characters on its input tape using Arabic notation will execute in asymptotically linear time, as demonstrated later in the section analyzing the increment operation. Its working footprint, however, will be logarithmic in space complexity, because that is how fast an Arabic representation grows with a count.

+

Because a Turing Machine can only reach another cell further out on the tape by stepping to it, space complexity and time complexity are related. A program that runs for ten steps can consume at most ten cells of tape. However, if that program merely bounces between two cells, it will require less space, precisely two cells. As another example, a program that counts the number of characters on its input tape using Hindu-Arabic notation will execute in asymptotically linear time, as demonstrated later in the section analyzing the increment operation. Its working footprint, however, will be logarithmic in space complexity, because that is how fast an Hindu-Arabic representation grows with a count.

If a program executed at the speed of a human operator, the operator would likely abandon the process before it finished. This highlights a necessary attribute of good software: utility. It also exposes a limitation of pure computation theory, which abstracts away physical time. Nevertheless, formal analysis remains a necessity. Consider an exponential-time program processing worst-case operands: its execution time explodes relative to input length, rapidly exceeding the age of the universe. In such extremes, empirical wall-clock measurement becomes superfluous. Computation theory classifies a program's behavior, which establishes structural implications for wall-clock time, rather than calculating absolute durations.

@@ -1282,7 +1282,7 @@

As a possible practical solution, note that if the program does not exhaust the current tape, the operator will never be called. How much tape is required to ensure this? An analyst could choose worst-case operands and measure the footprint when the program runs. At first, this appears to be the familiar 'my number is bigger than your number, I'll tell you mine after you tell me yours' game, which can be viewed as the definition for the countable infinity. However, there is a loophole. Running the system once with the worst-case operands to establish time and space ceilings guarantees the program can be allocated sufficient resources later for other operands. This is a practical approach, provided the program is a workhorse utility rather than an algorithm searching for a solution to an unsolved problem, and thus only needs to be run once.

-

Another practical solution, one that also applies to the first run of the program, is to analyze the logic to compute the time per step and extrapolate the total execution time. This is structurally viable only for specific programs; for others, tracing a path through the execution logic is as computationally complex as running the program itself, taking us back to the analysis in the prior paragraph. However, this phenomenon restricts itself to specific programmatic classes. Consider the previously mentioned examples of the ten step machine, the machine bouncing between two cells, and the Arabic counting machine. All of those programs can be, indeed were, analyzed before they were run.

+

Another practical solution, one that also applies to the first run of the program, is to analyze the logic to compute the time per step and extrapolate the total execution time. This is structurally viable only for specific programs; for others, tracing a path through the execution logic is as computationally complex as running the program itself, taking us back to the analysis in the prior paragraph. However, this phenomenon restricts itself to specific programmatic classes. Consider the previously mentioned examples of the ten step machine, the machine bouncing between two cells, and the Hindu-Arabic counting machine. All of those programs can be, indeed were, analyzed before they were run.

Suppose a controller is not analyzed to determine its computation theoretic complexity, or even tested against worst-case inputs, but is instead run with random or everyday input to gather performance measurements. This process is called profiling. After many runs, a programmer might surmise the behavioral limits of the program. However, many programs are neither linear systems nor smooth functions. With a different set of inputs than those used for profiling, perhaps even values adjacent to prior inputs, the program behavior can shift drastically. Consider the Pentium divider. It did not matter how many millions of times the result was accurate; the fact remains that customers later found inputs that yielded wildly inaccurate results. Only by elevating the analysis to the structural logic of the code can such an eventuality from an erroneous program be categorically ruled out. This is why the K5 transcendental function development project included a proof writing phase; see Thomas Walker Lynch, A. Ahmed, M. Schulte, T. Callaway, and R. Tisdale, "The K5 Transcendental Functions," Proceedings of the 12th IEEE Symposium on Computer Arithmetic, 1995. DOI: 10.1109/ARITH.1995.465368.

@@ -1524,18 +1524,18 @@

-

Binary Arabic representation address

+

Binary Hindu-Arabic representation address

- Would it be computationally consequential if binary Arabic representation were used instead of unary for addresses? For unary representation each increment was constant time, but with Arabic addresses, increments have a carry. Would the carry cause stepping to the right, combined with calling the address machine, to become slower and slower as the machine steps further and further? + Would it be computationally consequential if binary Hindu-Arabic representation were used instead of unary for addresses? For unary representation each increment was constant time, but with Hindu-Arabic addresses, increments have a carry. Would the carry cause stepping to the right, combined with calling the address machine, to become slower and slower as the machine steps further and further?

- For this discussion, the Arabic representation address is variable precision, so it will grow as the count becomes larger. The empty symbol will serve as the terminator. + For this discussion, the Hindu-Arabic representation address is variable precision, so it will grow as the count becomes larger. The empty symbol will serve as the terminator.

- For Arabic_increment, the machine reads the cell under the head; upon finding a 0 or the terminator, it writes a 1 and halts. Upon finding a 1, it writes a 0, steps right, and repeats the procedure. + For Hindu-Arabic_increment, the machine reads the cell under the head; upon finding a 0 or the terminator, it writes a 1 and halts. Upon finding a 1, it writes a 0, steps right, and repeats the procedure.

@@ -1632,7 +1632,7 @@

- Consequently, maintaining an Arabic address on a secondary Address Machine remains a constant time operation in the amortized sense, making it computationally inconsequential to the complexity class of the base machine. However, if a problem were to increment and decrement repeatedly around a binary power count, the behavior would be the limiting logarithmic time due to the length of the counter. An unlucky situation like this is called aliasing. + Consequently, maintaining an Hindu-Arabic address on a secondary Address Machine remains a constant time operation in the amortized sense, making it computationally inconsequential to the complexity class of the base machine. However, if a problem were to increment and decrement repeatedly around a binary power count, the behavior would be the limiting logarithmic time due to the length of the counter. An unlucky situation like this is called aliasing.

@@ -1664,7 +1664,7 @@

- In summary then, using Arabic representation for addressing is not statistically computationally consequential. On the other hand, the idea of using a decoded address in place of a tape head raises computational model issues. So much so that it would not be an exaggeration to say that the very goal of modern computer architecture is to ensure that, within an execution context, address decode does not become computationally consequential. Architects employ a memory hierarchy to do this, and that memory hierarchy rewards locality of reference, but access times are not linear in time relative to distance from prior access. Instead they are clumped into a constant time access abstract concept of locality groups where the time to access each group grows exponentially with the distance from the processor. + In summary then, using Hindu-Arabic representation for addressing is not statistically computationally consequential. On the other hand, the idea of using a decoded address in place of a tape head raises computational model issues. So much so that it would not be an exaggeration to say that the very goal of modern computer architecture is to ensure that, within an execution context, address decode does not become computationally consequential. Architects employ a memory hierarchy to do this, and that memory hierarchy rewards locality of reference, but access times are not linear in time relative to distance from prior access. Instead they are clumped into a constant time access abstract concept of locality groups where the time to access each group grows exponentially with the distance from the processor.

@@ -2371,6 +2371,7 @@ Now suppose defining a Turing Machine that initially has the head on the leftmos

An example of a non-Turing tape like model is the Emacs vertical line cursor model, where a cursor is said to be between characters. An ASCII file offers no such feature as 'in between' characters. Like a Turing Machine tape, a medial character in an ASCII file has a left neighbor and a right neighbor character. Any attempt to represent an in between cursor within the file itself would require inserting more characters into the file under the same model of every medial character having a left and a right character. Consequently, though emacs presents a model where cursor is located in between two characters, this model is only due to the interpretation of the functions' actual effects presented to users in the documentation. For example, instead of saying a cursor is located upon a character, and that inserting a character inserts the character to the right of the given character, the manual gives the description that the new character is inserted at the cursor location, where said cursor is in between the given character and its right neighbor. Thus the documentation presents the user with one model, which by necessity of using standard library calls to work with files, is built upon another model.

+ Computational Analysis @@ -2459,7 +2460,7 @@ Now suppose defining a Turing Machine that initially has the head on the leftmos

- If we were to dynamically evaluate these ASTs in the first order, the Arabic representation of each number would be physically written in unary code on the tape. + If we were to dynamically evaluate these ASTs in the first order, the Hindu-Arabic representation of each number would be physically written in unary code on the tape.

@@ -3233,807 +3234,275 @@ Now suppose defining a Turing Machine that initially has the head on the leftmos

This section reviewed finite difference methods as a means to implement function extensions. A person should notice that finite differences are one among many methods for extending functions, and they are not always the best choice. In modern computing it is not common to implement function extensions. Rather it is more common to increment a value across a domain, while completely repeating the evaluation for each new domain value, for example when computing values to place on a plot. However, independent of their importance in applied computing, function extension and difference methods are of theoretical importance for analysis, particularly when applied to nonstandard analysis. These concepts speak directly to the thesis of computational naturalism—deriving mathematics from computational logic—rather than merely proposing practical methods for implementing software.

- - - - Multiple machines and sharing tapes - -

Suppose we unmount a tape from a halted Turing Machine, say machine T0, and then mount the tape on another Turing Machine as input, say machine T1. Suppose we do this so that machine T1 may calculate the length of the output made by the first machine. When we do this, we run into some problems.

- -

Firstly, for a conventional Turing Machine, the tape that is mounted on T1 will be infinite, so no computational T1 machine will be able to process it unless there is a message on the tape telling T1 where the end of the input is. For our TTCA Machines, if T0 starts with a null tape, and then expands it, and T0 is computational, then at the time T0 halts, the tape will be finite. T1 can then use the continuation of stepping beyond rightmost to know that it has processed all of its input.

- -

If a Turing Machine does nothing then halts, it will implement an identity relationship between input and output. If we do not want the input given to a machine to 'bleed through', we will have to erase it. A computational TTCA Machine starting from a null tape can only produce finite tapes, so T1 can erase whatever T0 writes to the tape. However, if we are analyzing, instead of running the TTCA Machine, we might discover that the tape length would be infinite if the machine could be run. We know that if we pass an infinite tape to a computational TTCA Machine, it would not be able to erase the tape. Fortunately we can derive this fact through analysis and proof without having to run the computational machine to see if it erased the tape.

- -

Hence if we allow for infinite inputs, we should add an 'erase-to-end-of-tape' instruction to our machines, so that the computational machines may leave a tape with only their outputs on them. Our erase-to-end-of-tape instruction will be executed in a single step.

- -

Once an input tape is mounted, T0 is a computational TTCA Machine given a finite tape, its output is a finite tape, and that is taken to be the input for T1.

- -

T0 is required to provide meta information about the length of the active area. This approach is known as in-band signaling. Because length information is mixed with the data.

- -

T0 updates a dedicated table where it keeps track of the location of data objects and their lengths. Such a table is called a symbol table, and such a system is called a type system. This is written to a separate tape, or it is inserted at a known location on the main tape. In the case it is written to the front of the tape, the data following will have to be moved when the table grows. In the case it is written at the end of the tape, the table will have to be moved when the data grows. In either case the symbol table entry lengths themselves must also be managed, typically they are either coded into the controller or make use of in band signaling. It is conceivable that a symbol table would describe itself.

- -

We adopt a convention of maintaining a compact tape. As such we have no embedded empty-symbols in the active area. Then the empty-symbol marks the end of the active area.

- -

Each machine has two tapes, an input tape, and an output tape.

- -

A compact area is one that has only alphabet symbols (no empty-symbols). We can extend this concept to say that the density of an area is the ratio of alphabet symbols to empty-symbols.

- -

The approach we use in modern computing is often the 3nd one listed above, that of the type system. We carefully account for the length of each instance of data. Then we build up each larger instance from smaller ones, and while doing so, we add the lengths of the smaller instances to calculate the length of the larger instance. All programs then specify when they make or compose instances and their types.

- -

A system for creating the appearance of many tapes over a single tape is called a memory manager or a memory allocation system. It is typically better to use one of these and multiple tape models rather than solving the problem of moving objects around in memory in an ad hoc manner.

- -

Here is an important question for the TTCA Machine model: is it possible to define a memory manager that creates the appearance of multiple expanding tapes over the top of a single expandable tape? This problem is solved for managing files on a hard disk. However, all current file systems break at some point when pumped. So the question being asked here is equivalent to asking if a files system can be made without such architectural limits. The answer to this is yes, it is possible, as we will see in a later chapter of this book.

- -

Hence there are multiple conventions we may use for implementing the abstraction of multiple tapes on a tape machine. Multiple tape machine are equivalent in power to one tape machines, but sometimes they are easier to think about.

- -

Now as we have multiple tapes it is fairly easy to show that having multiple tape machines is equivalent to having one machine. We first view the multiple tapes worked on by the multiple machines as the multiple tapes of one machine. We then compose the state controllers in series, where the halt state of one is the initial state of the next one. Thus we may also conclude that having multiple tape machines is not more expressive than having one tape machine. We may also look at our multiple composed state controllers as one multiplexed controller, and then conclude that having multiple machines will not have an order of speed or space usage advantage.

- -

The advantage of having multiple tape machines each perhaps having multiple tapes is that sometimes it is easier to think about. It is a method of partitioning the problem.

- -

Suppose our machines have two tapes where one is called the input tape, and the other tape the output tape. Our procedure for passing tapes between machines will consist of umounting the tape from the source machine, then taking that tape over to the destination machine and mounting it as the input tape.

- + Unsigned integer representation -

Abstract areas and partitions

+

Charles Burnett published a very interesting book about the translation of Hindu-Arabic mathematics in Europe during the Middle Ages Charles Burnett, Numerals and Arithmetic in the Middle Ages (Farnham: Ashgate Variorum, 2010). DOI: 10.33137/aestimatio.v9i0.25990.. He describes a 9th-century treatise on Hindu-Arabic arithmetic authored by Muḥammad ibn Mūsā al-Khwārizmī, where "al-Khwārizmī" indicated he was from Khwarazm in Central Asia. When this manuscript was translated into Latin in the 12th century, European translators approximated his name as Algoritmi or Algorismi, thus giving us the word algorithm. He notes that the Arabs called such numbers "Indian Numbers". Another chapter, "Why we read Hindu-Arabic numerals backwards," shows that the little-endian and big-endian debate that has plagued programmers for decades has its origin in the Middle Ages Danny Cohen, "On Holy Wars and a Plea for Peace," IEEE Computer 14, no. 10 (October 1981): 48-54. DOI: 10.1109/C-M.1981.220208.

-

- A tape area and partitioning can be an abstraction defined by a function rather than merely by a leftmost and rightmost cell. Such areas can have different topologies than those of the base tape. A familiar example for most computer scientists is utilizing software to create the appearance of a two dimensional array over a linear memory. -

+

The information about ENIAC in this chapter is found in a 1947 IRE article, "Electronic Computing Circuits of the ENIAC" by Arthur W. Burks Arthur W. Burks, "Electronic Computing Circuits of the ENIAC," Proceedings of the IRE 35, no. 8 (August 1947): 756-767. DOI: 10.1109/JRPROC.1947.234265. I found it interesting that some of the design issues for flip-flops made of vacuum tubes resemble those of designing static RAM cells in CMOS. Although ENIAC's implementation is electronic, its architecture is fundamentally that of a mechanical machine.

-

- Accordingly, suppose there is a three tape Universal Turing Machine gasket machine that holds the definition of a base machine on a first tape. It calls the base machine as a subroutine to access the base machine's tape (the second tape), and it uses its own tape to organize the tape abstraction. Then this outer machine can present to its user a variety of transforms of the base machine's tape. -

+

Hindu-Arabic Numbers

-

- As an example, a gasket machine could partition the base machine's tape into two areas, one consisting of the odd addressed cells, and the other of the even addressed cells. Though these two areas consist of noncontiguous cells on the base tape, when viewed through the gasket machine, they appear perfectly contiguous. In this case, each area will have a leftmost cell and remain open on the right. -

+

An Hindu-Arabic number consists of a sequence of digits, d_0 d_1 d_2 \ldots where, in base 10, each digit has a value ranging from 0 to 9. In this section, these sequences are represented using sequence notation. For example, X = [7, 8, 9] is a sequence with three components. Its zero index component, x_0, is 7, etc. Note that X is written as a capital letter to denote that it is a container, while its individual members use small letters.

-

- When the base machine tape cells behind an abstract area are physically contiguous, we say that the abstract area is compact. In the odd even tape partition example, the abstract areas lack compactness. -

+

A person interprets these sequences as numbers by using a weighted sum.

- - Zero length is a second order concept +
+ a = \sum_{i=0}^n x_i \cdot ten^i +
-

Suppose we have a Turing Machine that is designed to mark an area based on some property of the symbols.

+

Here a is a numeric value, so it is written in lower case. Each x_i is the ith component of the sequence X. The value x_i is also known as the 'ith digit' of the number. The value ten is called the base or the radix of the number. It is fortunate that ten is so well known that it has a name, because otherwise an author is tempted to write the base while using the exact representation that he is trying to define. Although this function gives numeric meaning to our digit sequences, performing the suggested computation is pointless. The result is a number, and a person must represent that number, resulting in the exact sequence that was given as the input to the function in the first place.

-

Suppose further that starting with the head on the first cell, our Turing Machine will step right zero or more times until it finds a cell that holds a symbol that has the special property.

+

In many contexts, a person drops the container notation without causing ambiguity. Conventionally, dropping the notation reverses the order of the components. The sequence from the prior example, X = [7, 8, 9], becomes x = 987. To be clear, note that this number is 13 away from a thousand.

-

Once it finds such a cell it will write an area marker symbol to that cell, step right, and repeat writing area marker cells until it finds a cell that holds a symbol that lacks the area property. At which point the machine halts.

+

Independent of whether the number is represented with or without the bracket decoration, its most significant digit is the one with the greatest weight. In this example, it is the digit with the value 9, i.e., x_2. Analogously, the least significant digit is 7, i.e., x_0. Using a term from the previous chapter on arrays, the extent of our example number is 2. The digit count, or length, of this number is 3.

-

Once an area is marked, we can go back and run a length measuring machine that counts the sequence of marks.

+

Meaning of the word digital

-

However we have a couple of problems, if there is no cell on the tape that has the special property. The area marking machine will step right without halting while looking for the marker. As a second problem, if the all the cells to right of the leftmost area cell, the area marker machine will never halt. -

+

The first computing machines which performed discrete state computation made use of ten symbols for a digit’s potential values while mimicking how humans perform manual arithmetic. Examples include Pascal’s calculator, Babbage’s machine, the mechanical calculators that came after, Aiken’s Mark computers, and ENIAC.

+

Mechanical machines used physical gears with ten positions, each being 36 degrees apart. An index, such as a metal pointer, pointed at one of the marks, indicating which digit value had been registered on the gear. To change which digit value was registered, the gear turned until the index pointed at the mark corresponding to the desired digit value. Instead of gears, the electronic computer ENIAC used circular shift registers of ten vacuum tube flip-flops. These were called ring counters or decade counters. ENIAC operated on ten-digit signed numbers, so there were ten such ring counters plus a flip-flop for each number. It is structurally important that a person does not conflate the number of available digit values with the length of the digit sequence.

+

Mechanical machines, such as Howard Aiken’s Mark machines, gated rotational momentum with control linkages and clutches. ENIAC used an electronic analogy to the main rotating shaft: a central pulse clock. Pulses were sent to an electronic gate, and if the control to that gate allowed it, the pulses passed through to the corresponding ring counters to cause each flip-flop in the ring to flip in succession depending on the pulse count.

-

Now suppose we employ a second order analysis.

+

To add digit values on a mechanical computer, the two gears in question were mechanically connected. When one rotated back to zero, the other rotated up to the sum, possibly tipping a carry bar along the way. On ENIAC, one ring counter gated pulses to its partner while counting down. While pulses were gated to it, the second ring counter counted up, optionally setting a carry flip-flop.

-

Instead of running our area marker Turing Machine, we examine its definition and the definition of the machine that generated the tape data, to learn if such a machine will ever halt.

+

Hence the term digital initially meant both being discrete and making use of ten-state logic. If a person examines the control levers of the mechanical machines, or the control signals of ENIAC, he finds two-state switch logic, but computation units processed numbers consisting of ten-state digits, mimicking human computers. Early computing work gave us more than the techniques we take for granted today; it gave us words such as gate, register, and index.

-

Although we know that it is not possible in general to analyze machines to know if they will halt, or not, it is certainly possible to do so in some cases, and this is one of those cases.

+

Earlier, Claude Shannon and others wrote extensively on methods for simplifying switch logic, making it apparent that direct binary computation could be performed. The first electronic computer to do so was the Atanasoff-Berry machine developed in the late 1930s. Because there are two states in switch logic, it is maximally efficient to use a base two number system for arithmetic. In such an arithmetic system, each binary digit has the value of either zero or one. It is conventional to shorten the term binary digit to bit. John von Neumann assigned Atanasoff to audio work during World War II, so the Atanasoff-Berry machine remained unused in a basement at Iowa State University.

-

Upon analysis of our area marking machine we make a startling discovery: inputs exist for which the area marker will never halt.

+

Binary computation prevailed. By the 1950s almost all discrete state computers used two-state switch logic for computation. Still, the term digital continued to describe them. This leads to some curious naming conventions. For example, the company Digital Equipment Corporation, which began shipping computers in 1959, exclusively built binary computers. As another example, an engineer who studies modern digital electronics will never see wheels or decade counters used to implement operations in an ALU. Though the term digital continues to refer to computing with discrete state digits, the term has lost the connotation that there must be exactly ten of those states.

-

In one case, if there is no leftmost symbol with the property that defines the area we are looking for, the marker machine will search forever.

+

Today, computing based on ten-state digits, such as what is found on handheld calculators, is known as decimal computing. If a machine made use of 256-state digits, it would be neither decimal nor binary. Because we lack a special name for the number 256, a person states that such a computer uses base 256 digits and that it is a base 256 computer. Decimal computing, binary computing, and base 256 computing are all examples of digital computing. Independent of the base for computation, control signals are almost universally on or off. We say these are Boolean or binary valued, sometimes referring to their values as one or zero, but note they represent logical states rather than numbers.

-

In the second case, once the area is found, it is open on the right and it never ends.

+

As computer implementations moved from ten-state logic to two-state logic, it was difficult to completely abandon base ten number systems, particularly in business applications where users expect dollars and cents results that match manual computations, even when fractions and rounding are involved. Hence, IBM designed a scheme whereby groups of 4 bits represent the decimal digits; this code is called binary coded decimal or BCD. A machine which processes numbers using BCD has a decimal architecture.

-

We might say that inputs that have no leftmost symbol have an area of zero length.

+

Most computer users never see a memory dump. Instead, they see the output of print functions. By default, print functions render numbers in a decimal format familiar to the user, independent of the internal representation used for numbers. In contemporary computing, the time it takes to convert a binary number to a decimal number is negligible compared to the time required to execute a long computation, so decimal printing is practical.

-

This is an abstract concept, because our area marking and area measuring machines will never be run then halt and return a length measure of zero.

+

Scanning-Order and Digit-Order

-

Instead we might arrive at this conclusion that a non-existent area has zero length through inductive reasoning: Say we have an area of length i, and then remove one element from the area, then it has length i-1.

+

Had this book been written in Hindu-Arabic, the text would be written right to left. Let us explore what that looks like by using an example where we start with an English sentence and then reverse it. Notice that in this right to left string, the sequence [9, 8, 7] without notation is 987. Both strings match.

-

As we repeat this, then eventually we will have an area of length 1 as discovered by our length assigning machine. Now we remove 1 more element.

+
+ Writing from right to left the sequence [7, 8, 9] without notation is 987. +
-

Each time we removed an element before, it made the length smaller by 1, so we reason that 1-1 = 0. The area is now length zero.

+

When reversed this becomes:

-

We cannot go any further because there are no more elements to be removed.

+
+ .987 si noitaton tuohtiw [9 ,8, 7] ecneuqes eht thgir ot tfel morf gnitirW +
-

In this case we arbitrarily imagined a longer area. No such longer area was stated in the problem formulation.

+

Following the 1,000-year-old convention, I did not reverse the order of the digits of the number. For the reversed sentence to make sense, I changed 'tfel ot thgir' (right to left) to 'left to right', which is the sort of adaptation that Middle Ages translators did not universally apply. Also, I changed the brackets so that they still enclose the sequence components; otherwise, they point outward. When reading the number, the eye first lands on a digit of unknown weight. A person must scan further right to find the one’s place, then scan back while counting the digits to make sense of the number. Reversing the string is insufficient; a person must understand the sentence and change the phrase "right to left", to "left to right".

-

In real world programming, this is the difference between having a container that by implication will hold elements, and lacking the container entirely.

+

Now consider starting with the exact same right to left string as the early translators did, but instead of making an exception for numbers, we literally reverse the entire string:

-

By applying the limit logic, we have implied that an area exists.

+
+ Writing from right to left the sequence [7, 8, 9] without notation is 789. +
-

If running the area maker machine is asserting the existence of such an area, then it makes sense to speak of a zero length area.

+

Now both the scan order of the sequence and the digits are identical. This maintains the property designed into the original Indian Numbers. Note, that 789 here is still thirteen away from one thousand.

-

However, if we are not making this assertion, then there is no area so the issue of length is irrelevant. This is a nuanced problem.

+

To make the structure uniform, Western writers must either start writing right to left like the Arabs do, or reverse the order of digits in numbers. Computer architects implemented this latter option; for example, Intel processors utilize it.

-

Note, then, that zero length is a second order concept. We cannot mark nor measure it, and given our first order definition for an area, nor can we even represent such an area at a first order.

+

In summary, this section defines two concepts: scanning-order when reading, and the digit-order of numbers. Changing the scanning order without changing the digit order effectively causes the number to be read in reverse.

-

There must be some external structure present that implies the existence of the area for zero length to even make sense.

+

All conventional computers use low-to-high address scanning order for writing and reading text. Human operators universally find that low-to-high address scanning order for text feels natural. Relative to the low-to-high address scanning order, Big Endian computers follow the current Western ideal that the large digit comes first. In contrast, Little Endian computers are more faithful to the design of Hindu-Arabic numbers, storing numbers consistently with the scanning order matching the order of the digit weights.

-

This insight explains a lot of the pain related to the processing of end cases in computing. It will come up again in this volume when we examine the question of the emptiness of containers, see the section .

+

Because a page of text is two-dimensional, there are additional writing direction conventions. However, computer memory is one-dimensional, so only the two discussed here are relevant to this architectural discussion. Programmers tasked with writing display drivers encounter the others.

-

It also explains why loops so often need to be primed or given special case guards, which operates identically to adding a layer of analysis. We will discuss this further later when introducing the first-rest pattern see, .

+

Binary, Octal, Hexadecimal, BCD

-

It is interesting that non-existence has collided with zero length. This seems to be a contradiction, as something that does not exist should not have any length at all.

+

A person interprets a sequence of n bits as a number by using this function:

-

All of this happens at the second order, at the order of analysis. At this order we can make a distinction between an area that we have given a name to, and perhaps a location, as compared to an area for which we have done neither of these things.

+
+ \sum_{i=0}^n x_i \cdot two^i +
-

Thus for purposes of second order analysis we will say that an area exists if it has a name or a location, even if it has zero length.

+

As with sequences of decimal digits, a person drops the sequence notation to yield strings of bits. Similar to decimal digit strings, there are two options for writing the string: most-significant-digit-first, or least-significant-digit-first.

-

We will say an area does not exist if it has neither a name or a location. Again, execution of our first order area marking and length machines cannot provide us with any such information.

+

Although both strings of binary digits and strings of decimal digits grow in length logarithmically when counting, binary strings grow more than twice as fast. The expansion to a length two string occurs immediately at the count of two. Length expands to three at a count of four, and then to a string of length four at a count of eight. While incrementing to eight, the binary digit string has already expanded to length four, yet for the exact same count, a decimal digit string remains one digit long.

-

Need for the concept of cell

+

Writing such long strings is inefficient, so programmers group bits. When bits are grouped in threes, the base is eight instead of two. This is called octal notation, using one of the symbols 0, 1, 2, 3, 4, 5, 6, or 7 for the octal digit values.

-

Let us ask a question, what is it that an address is actually locating? Let us consider this question in the light of an example. Suppose we have the tape sequence of:

+ + + + + + + + + + + + + + + + + +
octalbinary
0000
1001
2010
3011
4100
5101
6110
7111
- - a, b , c, ε, ε, ε … - +

In the 1960s, computer panels featured switches and lights organized in threes, and coding forms were filled out in octal. Today, documents showing bit strings universally use groups of fours. Such a digit has 16 values, utilizing 0 through 9, and continuing with the letters a, b, c, d, e, and f. This is known as the hexadecimal system, often shortened to hex.

-

Now consider that we have an address of '2'. If we read tape address 2 we get back the letter 'c'. So the address is locating the 'c'. Now suppose we write at address 2. Say we write 'γ'. Now our sequence is:

+ + + + + + + + + + + + + + + + + + + + + + + + + +
hexbinary
00000
10001
20010
30011
40100
50101
60110
70111
81000
91001
a1010
b1011
c1100
d1101
e1110
f1111
- - a, b , γ, ε, ε, ε … - +

The table of hexadecimal digits is twice as long as the table of octal digits because each added bit doubles the table size.

-

Now we write 'Г', resulting in the sequence:

+

Another common grouping is the byte. This name is word play on the term bit. Today a byte is universally a group of eight bits; however, historical computers used other lengths, ranging from six to twelve bits. Vestiges of that past remain. K&R’s "C Programming Language" leaves the length of a byte unspecified. The designers of UTF-8 sought clarity regarding groups of eight, naming such a group an octet rather than a byte.

- - a, b , Г, ε, ε, ε … - +

When a group of eight bits, i.e., an octet, functions as a digit of a number, the arithmetic operates in base 256. Note that octet and octal are distinct concepts. An octet is a group of 8 bits, whereas octal is a number system based on digits that have values running from 0 to 7.

-

It would appear that the answer to our question is that address 2 is locating different things at different times. First it was, c, then γ, then Г. Yet the address did not change. It feels a little unsatisfactory to suggest that our concept of location depends on the value addressed. Also, notice, that when we made the Natural Number Turing Machine, that the values on the tape that was placed into correspondence with the Natural Numbers were never mentioned. Yet, we can't seem to answer the question of 'what is being addressed' without giving a value.

+

Groups of bits become larger. The organization of early RISC microprocessors specified that memory was always moved in groups of 32 bits, called words. Today, address variables are generally 64-bit words, while integer variables are either 32- or 64-bit words. Groups of bits found on internal buses scale larger yet.

-

Addresses speak to the structure of the tape rather than the values held on the tape. So as to facilitate this interpretation, we note that a sequence consists of a sequence of cells holding elements, rather than being simply a sequence of elements. Given the concept of a cell we can say that an address always locates the same cell, though the contents of that cell can change.

+

Bits are also grouped to construct numbers with bases that are not powers of two. In the BCD code, bits are grouped in fours to form decimal digits.

-

This operates similarly to a small extension to the already existing concept of a variable in mathematics. In mathematics we allow that a variable can take on different values, though its name never changes. Now we are going to say that a cell can take on different values, though its address never changes. Furthermore, as the cell is part of the sequence, we are going to say the cell itself has a left neighbor or right neighbor, excluding the value in the cell.

+ + + + + + + + + + + + + + + + + + + +
BCDbinary
00000
10001
20010
30011
40100
50101
60110
70111
81000
91001
-

Address of an area

+

In BCD, some possible bit value combinations are unused. This is an unavoidable consequence of emulating a number base in binary switch logic when that base is not a power of 2.

-

The address of a cell is the number of steps required to reach the cell when starting from the leftmost cell on the tape. The leftmost cell has an address of zero. It might seem intuitive to set the address of an area on the tape to be that of the first cell in the area. If we require that an area have at least one cell to exist, this approach works even for machines with a cell delete instruction.

+

Octal and hexadecimal operate primarily as writing aids. Internally, the computer computes in binary, i.e., base 2. In contrast, BCD computation relies on the structural encoding boundaries. When two BCD digits are summed, the carry occurs when the sum of two digits reaches ten or more, which does not fall on a power of two boundary.

-

Suppose that we are deleting the cells in an area. The delete instruction affects the cell to the right of the cell the head is on. Hence to delete all the cells of an area, the head is placed on the left neighbor cell to the area. Say that we delete an area of three cells. We will call delete three times. It seems unsatisfactory to say the area no longer exists after the three deletes because the head locating the area never moved. I.e. we still have a location marker for the area, so we could, for example, call append and put a cell back into the area. For the area to truly not exist, it would not have a location.

+

Some processors lack native BCD computation, providing BCD to binary conversion operations instead. Consequently, numbers that appear as BCD in memory convert to binary before computation. Other computers, most notably many IBM machines, implement true BCD architectures featuring instructions for directly operating on BCD encoded numbers.

-

Thus it seems more satisfactory to locate an area by its left neighbor cell, than to locate it by its leftmost cell.

+

Although BCD encoding is less efficient, BCD encoded binary place values numbers still grow in length logarithmically while counting. Because numbers are represented in base ten, no number conversion is necessary when printing, though today this performance cost is insignificant. More importantly, BCD numbers divide by ten without generating infinite fractions.

-

The inverse case also suggests that the cell to the left of an area defines its address. I.e. if we call append to grow an area, it grows to the right of the cell the head is on. According to this definition of area location, the rightmost cell locates a currently zero length right tail of the tape that will potentially be grown.

+

Allocation

-

In a sense what we are doing while exploring the meaning of area, existence, and zero length with these delete and append examples is performing a discrete limit in analogy to a limiting operation in calculous. To support having such a limit operation, the location of an area is that of the left neighbor of the leftmost cell in the area.

+

In most text documents, a number is written down once and remains unchanged. In contrast, a computing program frequently returns to the exact memory location and changes its value. This is equivalent to erasing an old value on a paper document and writing a new value in the same space.

-

Not all is satisfactory. When using the left neighbor of an area as the location for an area, we cannot locate an area that starts on the leftmost cell of the tape. If we make this a special case, then we have failed to make a first order definition for area, because the special case information will have to be stored in a higher level structure that describes attributes of the tape.

+

Consider recording a count in real time, where the system must routinely erase the current count value and write a new one. The process begins at zero. When an event occurs, the count increments to 1. The sequence remains a single digit. The count grows to two digits in length upon reaching ten. The count grows by another digit in length upon reaching a hundred, and again at a thousand. A digit sequence representing the count grows in length against the log of the count value. The log function grows without bound, but it does so slowly, allowing relatively short digit sequences to represent exceedingly large numbers. This structural observation is central to understanding contemporary computer architecture.

-

Nor can this use the left cell approach work in the case of multiple areas.

+

If a paper document only provides space for a single digit count, an error occurs when the count grows to ten, escalating when it reaches a thousand. If a programmer anticipates counts in the thousands, he allocates space for four digits in advance. Such an allocation minimizes wasted space because the digit sequences for massive numbers are not significantly longer than those for small numbers.

-

Suppose we have two adjacent areas.

+

For any allocated boundary, there remains the statistical possibility that a computed number will exceed the allocation, resulting in an overflow error. To establish rigorous allocations, a programmer must evaluate how much numbers grow via applied operations, and how many of those operations the system executes. This latter constraint ties directly to the projected lifespan of the task.

-
- Tape with two areas -
Figure 2 Tape with two areas
-
+

Universally, a program cannot analyze another arbitrary program and determine how many steps it will take before stopping, or if it will ever stop. Even in specific bounded cases, the analysis is highly complex, and the programmer frequently skips it unless the code drives a life-critical application. Because data lengths expand during runtime, users operate under the threat of allocation overflow. If the allocation overflows, there is a high probability the program will stop and emit an error, or it will continue executing while outputting garbage results. This failure mode does not surprise contemporary users. The real-time computer user experiences catastrophic failure, and the error emerges during accident analysis.

-

Here we show a first area, say a0, that includes cells 7, 8 and 9. And a second area, say a1, that includes cells 10, 11, and 12. Thus, by the convention of using the address of the cell to the left of an area as the area’s address, a0 has an address of 6, while area a1 has an address of 9. Because the areas are adjacent, the address of area a1 is the same as the address of the rightmost cell in area a0.

+

Conventional system memory is an array of allocation units called bytes. The indexes into this array are system memory addresses. Serving as the minimum addressable allocation unit defines the term byte. The number of bits in a byte is a parameter of the computer architecture, not the compiler.

-

We know that area a0 is located to the left of area a1 because a0's address is less than a1's. I.e. 6 < 9.

+

It follows that any larger allocation consists of multiple bytes. For a given allocation unit, the smallest address among the contained bytes establishes the address of the allocation unit. Each allocation unit is bounded by two parameters: its address, and its extent.

-

Suppose we delete cell 10. Though cell 10 is gone, our addresses remain consecutive, so what was cell 11, is now called cell 10, etc. and the diagram appears much as before, though area a1 is now only 2 in length:

+

Setting aside the scanning order variable, a number logically begins with its least-significant digit. This is justified because an analyst must read a number starting with its least-significant digit to derive the weights of the subsequent digits. The allocation scheme where the address of the allocation maps to the address of the least-significant digit is called little-endian. The definition of big-endian involves zero-padding to fulfill the allocation width. Relative to starting at the least-significant digit, the big-endian allocation address resolves to either the most-significant digit or the final zero in the padding.

-
- Step by step area two becomes shorter -
Figure 3 Step by step area two becomes shorter
-
+

In the architectural terminology utilized here, viewing memory as a horizontal tape with bytes in the cells and addresses increasing while moving to the right, little-endian numbers have the least-significant digit on the left, and big-endian numbers have the least significant digit on the right of an allocation. Stated structurally, little-endian numbers are zero padded on the right, and big-endian numbers are zero padded on the left.

-

Now we delete the cell at address 10 two more times, and all the area a1 is gone. During the deletion, and just after, the head will be on the rightmost cell of area a0, i.e. on cell 9. We can now say we have an area of zero length located at cell 9.

+

Figure 10 depicts a word featuring byte addresses represented in hexadecimal, running from c0 to c3. (In decimal these addresses represent 192, 193, 194, 195). The address of the byte before c0 is bf. The address after c3 is c4. The address for the word itself evaluates to c0, as it is the minimum byte address. This word holds a little-endian number. Treating a byte as an octet digit, the binary encoding for the least-significant digit of this number is 0001 1000. The most-significant digit is 1010 1110.

- Second area goes to zero length -
Figure 4 Second area goes to zero length
+ Figure 10: Left justified, least-significant-digit-first +
Figure 10: Left justified, least-significant-digit-first
-

Now suppose after deleting the area a1, as just described, we continue on to delete area a0. Each time we delete the leftmost cell of a1 the rest of area a1 moves left by one. Hence, after the first deletion the address for empty area a1 becomes 8. Area a0 is still located to the left of a1, because 6 < 8:

+

In Figure 11, the same number populates the word using big-endian architecture. For all but very large numbers, the digit pointed at by the allocation pointer evaluates to zero. A system continues to scan zeros until reaching either the end of the allocation or the most-significant digit. If it reaches the end of the allocation, the contained number evaluates to zero. Because this is the exact same number shown in the prior figure, it retains the identical least-significant digit and most-significant digit.

- Limit towards zero length being applied to the first area -
Figure 5 Limit towards zero length being applied to the first area
+ Figure 11: Same number using big-endian digit order +
Figure 11: Same number using big-endian digit order
-

Finally when all cells in area a0 have been deleted, a1 has collapsed into a0. Both have the address of 6, so the order between them can no longer be derived from looking at the base addresses. Should we attempt to reverse the steps above, and only be given the machine at its final state we would have to begin by guessing what the order was between the two areas, a0 and a1.

- -

So again, there is either missing information, or some information is not stored on the tape we are discussing.

- -

As yet another problem case that is either not possible or requires external information, if an area includes the leftmost cell of the tape, then its location cannot be described with an address.

- -

Hence this approach of using a discrete analogy to limits has led to some unsatisfactory end cases. We will visit this subject of areas on tape in the section, Area as a mapped tape and present a higher order approach for defining area location that avoids such end cases.

- -
- - -
- - - Some properties of Turing Machines - -

For a given Turing Machine, the input is the sole determiner of the output. I.e. each time the same input is given, we get the same output. The input and output are mathematical objects, hence Turing Machines are functions. However, it is common in computer science to speak of Turing Machines as 'solving problems' rather than saying they are functions. This is because we often think of the inputs and outputs of Turing Machines as being something other than mathematical objects. For example, when a Turing Machine sorts sequences found on its input, we might say that it solves a sorting problem.

- -

Turing Machines will differ due to differing alphabets, state controller graphs, associated instructions, start, and halt states. The choice of empty-symbol is inconsequential as long as it is distinct from the alphabet. As we will see in the later discussion on variations, the choice of alphabet is not very important. For two alphabets of the same cardinality we can setup a one to one mapping, and for those of differing cardinalities we can use sequences of alphabet symbols that map to alphabet symbols. For example, given an alphabet of ‘T’ and ‘F’, and a second alphabet of ’t’, ‘f’, ‘x’, ‘z’ we may make the following map:

- - - f - | FF - - t - | FT - - x - | TF - - z - | FF - - -

Then given this mapping, we may use two cells for each one cell for any {‘f’, ‘t’, ‘x’, ‘z’} alphabet machine, and then use the only the {‘T’, ‘F’} alphabet.

- -

There are a countably infinite number of permutations for alphabets, state controller graphs, associated instructions, start and halt states, hence there are a countably infinite number of Turing Machines that fit our definition. However, there are an uncountably infinite number of mathematical functions. Consequently, we must expect that some functions cannot be computed with Turing Machines.

- -

There might be multiple Turing Machines that perform the same function. A set of such machines forms a functional equivalence class. Within a functional class there will be a class of members related in that they all use the smallest number of steps when considered against the limit of input length. We discuss this further in the section on complexity.

- -

Of special importance to computation theory is the existence among all these infinite Turing Machines of a class of machines that read their state controller definition from the tape as an input. This is the Universal Turing Machine class.

- -
- - - Performance Analysis - -

An interesting aspect of the Turing Machine procedure is that it introduces the concept of stepping the machine. With the addition of some simple constraints it becomes possible to map the parts of the Turing Machine abstraction to the parts of some real machines. These constraints may take the form of such things as bounds on the length of the inputs, or the addition of out-of-resource errors. Because such constraints do not affect the 'normal' workings of the machine, the derived relationship between a Turing Machine step, and that of a unit of real time might not be that complicated. Indeed, except for some enumerable cases this relationship might even be so simple as to assign to a step an approximately constant amount of time.

- -

Because of the existence of a relationship between steps and time, particularly when it is a simple one, it is very interesting for us to know how many steps a Turing Machine will take.

- -

There are many ways to measure the complexity of a Turing Machine. Among these is something called the time complexity, which is a function that relates the length of the input to the number of steps required to reach the halt state.

- -

To derive time complexity we typically start with a step count formula which maps the length of Turing Machine input to the worst case largest number of steps. We then consider the behavior of this formula as input length goes towards infinity. To get this, we take the highest order term from the step count formula. Conventional results are constant time, polynomial time, or exponential time.

- -

We can derive the 'worst case length of the area written or read by the machine during computation' function in an analogous manner as for the step count function. This function is known as the space complexity. We may also consider the limiting behavior of this function to derive an order of space complexity.

- -

The order of time or space complexity will remain the same against certain variations of our Turing Machine definition. For example, if we double all the states by adding a second state that we always visit, where this second state does nothing, the number of steps would double but the functionality would not change. Order of complexity also would not change. A fixed time machine before doubling up on the states would still be a fixed time machine afterward. It is simply that the number of steps would be twice as large, but still a fixed number relative to the size of the input. A polynomial time machine would still be polynomial time, simply with double size constants. We say that changes which do not change order of complexity, nor existence proofs, are inconsequential.

- -

Suppose we have a complete Turing Machine functionality class. We say that it is complete because all possible machines for implementing the function are in this class. Some machines in this class will have a different order of time complexity than others. Now we consider the set of minimum order of time complexity machines from this class. As the larger set was complete, the set built against this constraint will also be complete relative to the constraint. We then say that this minimum order of time complexity is a property of the problem being solved, rather than being a property of a particular machine.

- -
- - - Conventional Turing Machine variations - -

In the first section of this chapter we gave a rather conventional definition for a Turing Machine. In the prior section, Complexity, we noted that we can analyze Turing Machines to find their time and space complexities. In this section we will discuss some variations that a person finds in the literature.

- -

A variation on the conventional Turing Machine definition is allowed when it can be proven that the variation never causes existence, order of time complexity, nor order of space complexity results to change, and in this respect is inconsequential. Earlier we gave the example of doubling up the states as being such a variation, though that is an excluded variation typically.

- -

Open in both directions tapes

- -

Some Turing Machine descriptions describe a tape with no end in either the left or right directions, i.e. a tape that is open in both directions.

- -

This feature adds no richness of expression, because we can get the same behavior from a Turing Machine with a single ended tape. To do this we partition the single ended tape into odd addressed cells and even addressed cells. The odd cells are said to be the right side of the tape, and the even ones the left side. We then rewrite any tape controller based on a bidirectional tape to instead use the 'odd' and 'even' channels instead of the left and right sides of the tape.

- -

This same approach form can be used to show that multiple tapes, or even multi-dimensional tapes, add no expressive power. The good news is that such variations can be used whenever convenient, and we will get the same results.

- -

Going in the other direction, the open in both directions tape is not a simplification. There is still a start cell, being the cell that the head is initially placed on. And as noted above, the topology around this start cell is no different, it is simply a question of the adjectives we use for describing it.

- -

Alphabet replacement

- -

Without loss of generality, we may replace the alphabet with a single symbol, say 's' (short for successor). This is because symbols in any alphabet can be placed into correspondence with a sequences of 's' symbols. For example, the symbols of the alphabet of {w, x, y, z} can be placed into one to one correspondence with the sequences in the manner of {<w, s>, <x, ss>, <y, sss>, <z, ssss>}. We will need to put the empty-symbol between any such sequences on the tape, so that two sequences can be distinguished from one longer sequence.

- -

Alternatively we can adopt a two symbol alphabet instead of a single symbol, where the second alphabet symbol is an end of sequence marker.

- -

In contemporary computing we use an alphabet of two symbols, {0, 1}, and fixed length sequences. No end marker is needed when the sequences to be placed into correspondence are of fixed length. So for example, the symbols in the alphabet {dog, cat, mouse, fish} may have the correspondence of {<dog, 00>, <cat, 01>, <mouse, 10>, <fish 11>}. Conventional fixed sequence lengths are 8, 16, 32, and 64. So for example, when the sequence length is 8, any alphabet of 256 symbols or less may be placed into correspondence. A conventional correspondence table is that of the ASCII code.

- -

The fixed length to be used can depend on computational context. (In contrast, UTF8 does not use fixed length sequences, so there must exist at least one end of sequence marker.)

- -

Another alternative to end of sequence markers for variable length sequences is to externally account for sequences lengths. We call such an accounting system a type system.

- -

Leaving out the empty-symbol

- -

When we use a fixed sequence of {1, 0} to stand for symbols, it is expensive to reserve a sequence for the empty-symbol. This expense is due both to losing the use of a symbol in the alphabet, and in the complexity of control circuitry when keeping track of it.

- -

The empty-symbol functions as a property of the machine rather than simply another alphabet member, because the tape initially has an infinite tail of empty-symbols. A computational Turing Machine is limited to taking a finite number of steps. Thus it cannot compute a tape initialized with an empty-symbol (or any other value).

- -

However we can add a constraint on all Turing Machine controllers that requires of controllers that they never write the empty-symbol, and always write an alphabet symbol to a cell before reading it. Then, because a cell is never read before being written, it does not matter what we write into it for initialization. We may even use an alphabet symbol. Consequently this constraint allows us to eliminate the empty-symbol. This gives us the following Turing Machine variation:

- - - 1. fixed finite alphabet - 2. fixed write before read constraint - 3. variable single ended tape - 4. variable read/write head - 5. variable read value buffer - 6. fixed controlling state machine - 7. fixed left from leftmost error - 8. fixed start state - 9. fixed halt state - 10. variable current state - 11. fixed procedure for using these. - - -

To validate that this is an inconsequential Turing Machine variation, rather than a description of new abstraction that is not a Turing Machine, we must show two things: Firstly, that any of the now disallowed controllers never compute something that cannot be computed in the presence of the constraint. And secondly, that there are same complexity class alternatives for any disallowed controller.

- -

No need to step into the inactive area

- -

1. When attempting to step into the inactive area, instead keep a counter for the number of steps the machine would take. Only allow reads or writes or head movement when the counter is no longer needed due to the head having moved back into the active area.

- -

2. Write an alphabet symbol and change the inactive area traversed into an active area.

- -

The TTCA Machine variation

- -

The active area on the tape can grow at most by one unit for each machine step. This largest growth occurs when the Turing Machine steps right and writes an alphabet symbol in every visited state. This means that for computational machines that start with a tape that has a finite input (active area), the output (active area) will be finite. This also means that space complexity can never be larger than time complexity.

- -

A fixed value is one that is provided with a Turing Machine definition, and does not change while the machine runs. Suppose we chose a fixed length Turing Machine tape. The tape would then have a rightmost cell. That cell would have no right neighbor, but would have a left neighbor. We would also add another error, that of right from rightmost. This error would be invoked when the controller attempted to step right from the rightmost cell.

- -

Consider a machine that does not step out of the active area, has constant space complexity, and where this space complexity is less than the fixed length for the finite tape; such a machine would never trip the right from rightmost error, and thus there would be no difference between a finite tape and an infinite one.

- -

Now suppose that we bound the length of the input, and that the maximum space required for such inputs or shorter ones is less than or equal to the length of the tape. Then again, the right from rightmost error would never be taken, and thus the tape would be indistinguishable from an infinite one. (Today we typically pad programs with lots of memory and long address words in hopes this will be the case.) An analogous argument can be made if we bound the number of steps that may be taken.

- -

Now consider the case where we do not fix the length of the input, nor the number of steps allowed, and that space complexity is such that space usage grows with growing input length, at least for very long inputs. For such machines we can always find an input of sufficient length to trip the right from rightmost error.

- -

As another approach to finite computing we can run computations twice. For a given input we first run the Turing Machine variation that does not step out of active area but still has an infinite tape. We watch this machine closely while it is running and find the bound on the active area. Now we can make a second machine that has a fixed length tape at least as long as our active area measurement but is otherwise the same. Now with this second machine we can run the same input and there will be no right of rightmost error, and thus there will be no difference between having the finite tape or an infinite one.

- -

Unlike for the constant space complexity proposal, and the bounded input length proposal, which only work for small subsets of potential inputs, this 'run twice' proposal derives a finite machine that works for any given input which a Turing Machine works for. Though, unfortunately, in all cases the second run will be moot, as we could have simply taken the output from the first run.

- -

In a variation on the run it twice approach, instead of running the first machine, we might instead analyze it and should we be able to surmise a maximum tape length, we could use that.

- -

In yet another approach we can extend the finite tape as needed. We place a control layer over the finite tape. When a step right instruction from the Turing controller invokes the right from rightmost error, the lower layer allocates memory, lengthens the tape, and then performs the requested step right. As long as this occurs in fixed time, (or of sufficient lesser order time than the dominate order of the time complexity), and as long as there is indeed more memory to allocate - this Turing Machine variation will yield the same order of computational complexity as one with an an infinite tape.

- -

I propose the following Turing Machine variation:

- - - fixed finite alphabet - write before read constraint - variable and extendable finite tape - variable read/write head - variable read value buffer - fixed controlling state machine - fixed left from leftmost error - fixed right from rightmost error - fixed start state - fixed halt state - variable current state - fixed procedure for using these - - -

In our original Turing Machine model, the controlling state machine instructions were limited to, do-nothing, left, right, write, with reading as an implied instruction. To this list we add append. The append instruction may only be called when the head is on the rightmost tape cell. This is not limiting because the instruction may be called from a state that is at the end of an arc triggered by the right from rightmost error. When we have no empty-symbol, append is given an alphabet symbol and performs a write into the new cell. This is not limiting because if need be, a person can always perform an extraneous write of an alphabet symbol.

- -

With this extendable tape model all Turing Machine components remain finite during computation, though some are arbitrarily large. This variation is more suited for creating a mapping between a Turing Machine and a real program running on a real machine. Specifically, a computation requiring N cells maps exactly to a physical machine possessing at least N cells. The theoretical model and the physical implementation remain perfectly isomorphic provided the physical environment does not exhaust its memory bounds. Because a computational TTCA machine demands strictly finite tape extensions, a physical machine that completes the execution maintains exact structural correspondence. This isomorphism holds continuously as long as memory remains available, even accommodating dynamic hardware expansion such as memory hot-swapping. The theoretical correspondence fails solely upon a hard physical limit being reached during execution.

- -

More about instructions

- -

The Turing Machine state controller has a instruction symbol tied to each state. The Turing Machine procedure then has us take action based on this symbol. This is our current instruction set:

- - - right, causes the head to move to the right neighbor cell. - left, causes the head to move to the left neighbor cell. - read, returns the symbol under the head. The returned symbol is then used to chose the transition arc. - write(x), causes the symbol x to be written to the cell that the head is currently on. The symbol x is any symbol from the alphabet. - append(x) may only be called when on the rightmost cell. Extends the tape by one cell, and writes the symbol x into that cell, where x is any symbol from the alphabet. - - -

We are going to relieve the constraint that append can only occur from rightmost. Our new append is functionally identical to adding a cell to the rightmost extremity, and then shifting all the symbols over by one cell starting at the new cell and ending when the new rightmost has been written - and then doing the requested write of x on the right neighbor cell.

- -

We will also include the inverse function for append. delete(append(x)) reads x while deleting the cell that x was in. The current Turing Machine model can emulate this function by shifting all the symbols in cells the right of the head left by one, and then simply not using the rightmost cell.

- -

We are also going to support multiplexed state controllers. Our multiplexed Turing Machine will have multiple heads. One for each separate thread of execution through the state controller. As explained in the following chapters, supporting multiplexing makes our machine more complex, especially in the presence of the delete instruction. However, we defer that discussion to the relevant chapters.

- -

In cases where successive states are visited in a fixed order it is convenient to combine the instructions. We have developed the concept of a instruction statement to support this:

- - - statement::[direction]instruction+[modifier][&contract]*[arg]* - direction::- | ε - instruction::r | w | s | a | d | m | e | ☥ - modifier::◧ ◨ n - instruction - r read cell under the head - w write cell under the head - s step - a allocate/add/append a new cell - d deallocate/drop/delete cell - m move, no allocation or deallocation of cells, requires fill - e entangled copy - ☥ entangled copy on a new thread - modifier - ◧ operate on leftmost - ◨ operate on rightmost - n repeats n times, n provided through an argument - contracts - h◧ head is at leftmost - h◨ head is at rightmost - examples - -s ; step left - a◧ ; make a new leftmost cell - a◨ ; append to rightmost - sn ; step n times - to derive a longer instruction, combine them: - as ; append then step - -a-s ; append to the left, step to the left - - -

The left direction is specified with a minus sign, otherwise the direction is taken as right going. So the letter s is the right instruction, and -s is the left instruction. The instruction s3 steps right three times.

- -

The instruction a appends and writes a new cell to the right of the head. We use two special characters from the UTF character set to signify the rightmost and leftmost of the tape. This one looks like a little tape with its left cell inked in, ◧, so we use it to stand for leftmost. We use this one, ◨, to mean rightmost. Hence a◨ makes a new rightmost cell, and a◧ makes a new leftmost cell.

- -

In some cases it is possible to implement higher performance implementations for instructions when the programmer tells us some additional information. For example a◨&h◨ has identical functionality as a◨, while the programmer also guarantees that we are on the rightmost cell. This saves the function from having to scan to the end of the tape.

- -

We can concatenate the instruction letters into a string to summarize what would happen sequentially in adjacent state transitions. If these compound instructions need arguments, then they are pulled from the argument list in order as they are needed. For example, as means to append, with the parameter for the append taken from the argument list, and then to step.

- -

We support multiplexing with the instruction e, which is short for entangled. This operator returns what appears to be a second independent machine, but this apparently independent second machine actually shares the same tape with the first machine. It is functionally identical to giving one machine two heads, and thus the ability to have state sets.

- -

The instruction esr is a compound instruction referring to sequentially applying three other instructions. The e says to make an entangled copy of the head. The s says to step this copy, and the r says to do the read. The analogous esw does a write as the last step. This sort of combination of letters to make more complex instructions was inspired by Lisp's car and cdr compositions. Though this is functionally what the instruction does, its actual implementation might be completely different.

- -

The combination of multiplexing and cell deletion leads to a hazard where one thread can delete a cell the head is on in another thread. We add a collision error continuation to our multiplexed Turing Machine interface because of this situation.

- -

Chapter discussion

- -

The modifications to the Turing Machine leading to our TTCA machine were inconsequential, in that for order of complexity and existence proofs we may swap one machine for the other and the results will be the same.

- -

The original Turing Machine had an infinite tape. In contrast the TTCA machine has a surprising property: for computational problems all of its components remain finite. This follows from the fact that during computation a machine makes a finite number of steps, so the tape can only be expanded to be a finite size.

- - - Consequentiality across the design abstraction stack - +

If the specified word holds a count, the counting mechanics differ. When counting with the little-endian convention, a number grows into larger addresses as the count carries into new digits. In contrast, with big-endian architecture, counting carries into strictly smaller memory addresses.

-

Choice of realization

+

Typically, processor registers and buses are one word in length, and a processor loads the entire contents of a memory word into a register as a single atomic operation. Contemporary processors are designed effectively for either little-endian or big-endian number representation; however, once designed for little-endian, the processor is fundamentally incompatible with big-endian, and vice-versa.

-

- Let us take the example of adding two Arabic representation numbers. Logically this is considered to be a logarithmic time problem. We break the operands into fixed length pieces, and adding them in pairs results in a carry per block. By recursively pairing the blocks and applying the carries, we generate wider carries. Thus we can show that in terms of the logic gates that must be traversed, the sum is a log time operation. -

- -

- Physics comes to a different conclusion. In the worst case, a carry into the least significant bit can affect the sum bit some physical distance away. As the operands get longer, this distance grows in proportion. So given the propagation of information at a fixed speed, the bounding evaluation time against growing operand width is linear time. Even if it is log time in gate count, at some point the interconnect delay will dominate. -

- -

- The logical analysis of the adder given above allowed for unbounded resources, because as the adder operand increases in size, the number of block adders increases without bounds. In any realization there will be a limit on the number of blocks that can be added in parallel. These groups are then processed one by one, and the carry is propagated between them. Consequently, as the operands grow in length without bounds, the adder evaluation time becomes proportional to the number of groups processed. Processing groups in series is a linear time algorithm. -

- -

- It is notable that the time-multiplexed use of computer resources produces the same linear time result as the physics of information propagation analysis for the adder. -

- -

- A Turing Machine program faces a situation analogous to physical constraints. Given the operands are found on the tape, and the carry-in can affect the msb of the sum, the head will have to move ever more cells rightward to convey that lsb information up to the msb. Based solely on the propagation time of that information, addition is found to be a linear time algorithm. This propagation remains computation class limiting even if the Turing Machine is given an unbounded number of independent heads. -

- -

- There appears to be alignment among physical limitations, resource limited computing, and steps spent by Turing Machines while they carry information across a linear tape. This alignment indicates that a reasonable realization will be computation theoretic inconsequential. -

- -

- At this point we have arrived at questions of the physics of computation. The above analysis made use of classical physics. Perhaps realizations based directly on principles from Quantum physics models will be computation theoretic consequential. -

- - -

Choice of implementation

- -

- From Babbage's Analytical Engine of 1842 up to the transition to mechanical relays and vacuum tubes in the 1940s, calculating machines were implemented with gears. The basic principle is apparent to anyone who has seen a mechanical odometer. Consider adding numbers for example: given two odometers, step one back at the same time as stepping a second one forward; when the first one reaches zero, the second will hold the sum. This process can be optimized, but the general idea remains the same. For such machines, a step is a rotation of the main shaft. -

- -

- The Harvard Mark I machine had a main axle speed that maxed out at 3000 RPMs, say 2700 RPMs to keep our math simple. Then this is 2700 steps per minute. The ENIAC was a similar implementation, but one that called out the use of circular shift registers of vacuum tubes instead of mechanical gears. Because there were 10 tubes in a ring register, it took 10 clock ticks to complete one 'rotation'. The clock rate maxed out at 450 kilohertz. That would be one rotation every 1/45,000 of a second, or 2.7 million RPMs, an improvement of three orders of magnitude! -

- -

- Yet, the same program when run on the Mark I took the same number of steps as on the ENIAC. But more importantly, a linear time algorithm on the Mark I was still a linear time algorithm on the ENIAC, etc. Thus, these implementation differences were computation theoretic inconsequential. -

- -

- It feels unsatisfactory to leave out the tremendous difference in clock rates. So let us address this feeling by naming an ENIAC main shaft 'rotation' as a standard 'step'. If we do this, then a Mark I shaft rotation would be 1000 ENIAC steps. Yet, this would merely affect the linear constant in the step count formulas. The same programs can be run, with the same inputs, and asymptotic behavior is the same for both machines, because computation classes do not include the constants on the step count equation. Constant time remains constant; linear, polynomial, and exponential time classes are the same as before. Programs that decide questions would get the same answers when they completed. -

- -

- Because we made an ENIAC shift register turn completion a 'standard step', we have a relative measure, so there is something we can do to create a computation theoretic consequential difference. Suppose we have two ENIAC machines, and we send one speeding away from Earth at an exponentially increasing rate, i.e. increasing red shift, and we observe it from Earth. We will observe that the clock on the traveling ENIAC is growing ever slower, and that a linear time program running on it will be observed to have exponential time behavior. Unfortunately, relativity does not smile upon us, as the people on the spaceship would not see the inverse, a speeding computation on Earth, but rather they would also observe a slowing one. -

- -

- So then, instead we send a spaceship towards Earth, with increasing blue shift, and we would observe that spaceship's ENIAC getting faster and faster. This is still not a computation theoretic speedup, because it is not asymptotic. In finite time, said spaceship would run into Earth, or pass it by and then be red shifting. -

- -

- A designer could purposely slow the clock on a second ENIAC so as to emulate red shift. For this to be more than mere theater there would have to be physical reason to run a slower clock than necessary, for example perhaps for conserving an ever dwindling battery. But slowing computation down, or even stopping it, is typically not useful. However, going the other direction, an ever faster clock does not work, as there is a finite maximum physical clock speed. -

- -

- We get an increasing blue shift situation with Moore's law. If every generation transistors become exponentially smaller, and thus faster, and we consider step times in years, hopping from new realization to new realization, then indeed linear time algorithms on a single realization would be log time algorithms on the generational computer. But chances are this is not an asymptotic, i.e. limit to infinity, phenomenon either. -

- -

Choice of organization

- -

- Superscalar and VLIW computers execute multiple instructions in parallel. Real data dependencies put limitations on how many instructions are available to be executed in parallel, but even discounting this, if a program were executed N instructions at a time, its time to execute would divide by N. This merely affects the linear component of the equation mapping step count to input length, and thus does not change the computation class. Superscalar and VLIW architectures do not affect decisions; indeed they are transparent to programs, so decider problem results cannot change. Hence these techniques are not computation theoretic consequential. -

- -

- In general, by definition, organizations do not change a program's view of the machine, as that is part of the architecture. So organizations will not affect decider results. Also, the memory operations will be the same, as that is viewable state, so space complexity does not change unless time complexity changes. -

- -

- The realization sets fixed resources, so any attempt at parallelization will be bounded, as in the superscalar and VLIW discussion above. Thus at best it can divide the execution time by N. -

- -

- Some organizations can arrange computation in a manner that the base clock can run faster than for other organizations. However, clocks run at a fixed maximum speed. On modern systems they can slow down to reduce heat dissipation or battery consumption, but that does not make programs faster. So if one organization has a faster clock than another, the ratio is merely a linear term contributor. Apart from stopping, there is nothing a clock can do to participate in the decision making of the program. -

- -

- Caching of values sent to the system memory again does not participate in the decision making of a program. We are at best looking at improvements in the linear term. -

- -

- Branch prediction saves the time required to do a full fetch, but fundamentally it does not change the data flow graph of the program. The same decisions are made. -

- -

- Suppose that an organization keeps the operands for a function in a content-addressable memory. When the operands are recognized, it then immediately returns the looked-up value. This approach, called memoization, bypasses the internal looping of the function. Hence, this does participate in the decision making of a program, and could potentially change the computation theoretic complexity of programs with certain properties where the same operands occur in patterns and lead to an expensive computation. Memoization is typically designed into computer languages, rather than being built into the organization of a computer. Chances are, at the computer organization level or lower, it is more efficient to simply execute the presented instructions. -

- -

Choice of architecture

- -

- Common decisions made at the architecture level are those for supporting RISC or CISC, the bit layout and handling of operands, the size of the internal register file, how DMA is to be handled, whether to use memory-mapped I/O or have explicit instructions for it, how interrupts are to be implemented and the number of entries in the interrupt table, what special registers are present and what features are available through them, how virtual memory and its user and process IDs are to be implemented with the possible use of a translation lookaside buffer, what onboard execution units will have direct instructions, the built-in data types, questions of unaligned accesses, bus standards to be supported, if sleep modes are to be present, how the machine will get booted, the security rings that will be supported, details of the hardware virtualization layer, special support for the OS, how the system stack will be handled, potential partitioning of address space, support for large buffers, and memory sharing features: none of these are computation theoretic consequential. -

- -

- As architecture enters the gray area with organization, cache architecture, bus layouts, bus buffers, direct inclusion of write buffers, perhaps a stack cache, prefetch buffers and split-transaction buses: none of these are computation theoretic consequential either. -

- -

- Said features certainly affect performance, but none participate in the decisions the program makes, change the number of execution steps by more than a linear ratio, or alter the memory complexity of the program. -

- - - - -
- - - TTCA Turing Machine in Lisp +

If an unconventional processor loads numbers sequentially as digit streams, the load instruction must possess a mechanism to detect the end of the loaded number, or it must receive the exact length prior to execution. This structural boundary is identical to the existing mechanism for loading character strings, where systems utilize length counts or explicit end terminators.

-

Because our TTCA Turing Machine has finite sized components, we may create a software model for the TTCA Turing Machine without having to make assumptions of the sort that ‘very large approximates the infinite’. Rather we can show a one-to-one mapping of code and data in the software model and the TTCA variation of the Turing Machine. Consequently, the software model and theoretic model are isomorphic.

+

For serial computation utilizing little-endian notation, the processor produces the first digit of the sum immediately after receiving the first digit of the operand. If big-endian is used, the least significant digit arrives last, forcing the system to buffer the entire number before computing the first digit of the sum. Attempting to load from the far ends of the operands fails because the physical location of the sum's far end remains undefined until the carry propagates. Resolving big-endian serial addition requires implementing signed digit arithmetic.

-

It follows that we can use our TTCA Turing Machine software to make theoretical statements about computation in the form of programs. By following this path we will learn some interesting things in this chapter about analysis, the meaning of emptiness, data type, the properties of non-destructive vs destructive programming styles, and multi-threaded programming among other things.

+

Bit Order Within Bytes

-

The Turing Machine, and our variation of it, may be partitioned into two parts. One part of the part consists of the tape head and the tape. We call this the Tape Transport Unit, as that is the name used for the mechanism that does this work on real tape storage units. The other part of the part is the Controller.

+

Data transports between hardware nodes over bundles of wires called buses. Bus specifications explicitly define the order of bits in bytes, and for contemporary machines, bytes are strictly octets. All compute processors, channel processors, and attached devices must conform to the bus’s specifications.

-

The Tape Transport Unit is given instructions for reading, writing, and moving the head. (In real Tape Transport Units the head is in a fixed position and we move the tape, but the relative affect is the same.) Our modified model adds instructions for extending the tape. During normal operation these instructions only come from the controller.

+

Channel processors bridge the gap between a computer's system bus and a storage bus connected to storage devices. The storage device designers adhere strictly to the storage bus specification without evaluating the host computer architecture. The compatibility burden relies entirely on selecting a channel processor that respects the established standard. Consequently, unless a programmer explicitly designs bus standards or hardware interfaces, the physical bit order within bytes remains invisible.

-

We have two types of controllers. One type of controller is a state machine. Its design is an integral part of the Turing Machine. To step the Turing Machine means to step this state machine to its next state. The state machine definition exists before the Turing Machine takes its first step, and its definition remains intact for as long as said Turing Machine exists.

+

Some processors physically store bits into memory bytes in different orders. However, the data read and written into the respective memories routes through a bus, and at that boundary, the bit order aligns with the applicable bus specification.

-

When we speak of a Turing Machine without adding further words to the term as qualifiers, we mean that it uses this type of controller. For clarity we can call this a Directly Controlled Turing Machine.

+

On all contemporary computers, ALU hardware performs arithmetic on bytes. Carries propagate strictly from lower significant bits to more significant bits; therefore, the ALU imposes a logical bit order within bytes. That logical order conforms directly to the documentation for the processor’s internal bus, where bit 0 serves universally as the least significant bit.

-

The second type of controller is the Universal Controller, and a Turing Machine that uses this type of controller is called a Universal Turing Machine. The Universal Controller reads the definition of a state machine controller off of the tape. Hence the Universal Turing Machine may emulate any Directly Controlled Turing Machine. We can also call this an Indirectly Controlled Turing Machine. Here the qualifier ‘Indirectly’ is intended in the sense it is used in assembly languages, meaning the data is not present in the controller, but rather it must be fetched from memory.

+

Because doing so is impossible in all but special cases, an address points at the entire byte, rather than pointing at the most significant bit or the least significant bit.

-

With our Tape Machine library, your program is the controller, and the library implements the Tape Transport Unit. (In the future I will rename this library to ‘TTU’ from the current ‘TM’. Also the use of the term ‘function’ will be changed to ‘routine’.)

+

Byte Order Within Words

-

Your program that uses the TTU library is in a sense direct control, because the program is already defined before the first step of the machine, and with the possible exception of self-modifying code, it does not change until the program exits, and thus the process no longer exists. On the other hand, your program is loaded from memory by the processor, thus demonstrating that the processor is an indirect, i.e. universal, controller. Either interpretation works depending on how broad of a view we want to take.

+

A specific processor architecture features native support for byte data alongside varied word lengths, most commonly 16, 32, and 64 bits (or 4, 8, and 16 bytes).

-

Installing the library

+

Communication channels and storage devices organize payloads strictly as octets, possessing no native structural support for words. Therefore, storing or transmitting a word requires serializing the word into a sequence of bytes, transferring the data, and subsequently reading the data while deserializing the byte stream back into the original word configuration.

-

TM is presented as an iteration library on the de facto package manager for Lisp quicklisp. Alternatively, a person may clone the repository www.github.com/Thomas-Walker-Lynch/tm and then checkout the latest release tag, which as of this writing is v0.7-alpha. After installing the code cd into the tm directory and run your lisp interpreter. Inside your lisp interpreter type the instructions (load “load”) and (test-all). test-all should return with a message that all of the tests passed.

+

Our Indian Number derived representations consist of sequences of digits. Words of allocation consist of consecutively addressed bytes. Hardware manages bytes atomically. Any bit encoding for the digits of a number must pack cleanly into bytes, otherwise the digits fracture. Achieving a clean packing requires padding the data with zeros to force alignment to an 8-bit boundary. When a system meets this criterion, treating a number as a sequence of bytes acting as digits maintains structural consistency. Due to this constraint, little-endian and big-endian are routinely classified as byte orders.

-

It is possible that the threading tests, 'ts1-' might fail if your machine is heavily loaded or very slow, as they have timing built into them, but this is unlikely. Then type either (use-package :tm) or (in-package :tm) depending on what your objectives are.

+

The following figure displays a stream of bytes arriving as data and being copied into a word. The digits of the word (the bytes) arrive in little-endian order and target a little-endian machine, so they are written in the exact order they are scanned off the channel.

-

The examples in this chapter either come from the tm/test directories, or from the tm/docs/examples directory. At the time they were placed in the book, they executed correctly, and I have endeavored to keep the examples up to date.

- -

This is what it looks like when I follow install using the git clone method:

- - - > git clone http://www.github.com/Thomas-Walker-Lynch/tm - Cloning into 'tm'... - remote:Counting objects: 3052, done. - remote:Total 3052(delta 0),reused 0(delta 0),pack-reused 3052 - Receiving objects: 100% (3052/3052),2.41 MiB|1.52 MiB/s,done. - Resolving deltas: 100% (2366/2366),done. - > cd tm - > git tag - v0.1-alpha - v0.7-alpha - > git checkout v0.7-alpha - Note: checking out 'v0.7-alpha'. - You are in 'detached HEAD' state ... - > sbcl - This is SBCL 1.3.14.debian ... - * (load "load") - ; compiling file "/home/tm/package-def/conditions.lisp" - ; compiling (IN-PACKAGE #:TM) ... about 10 pages of these - hooking test: TEST-TS1-5 - T - * (test-all) - + TEST-UNWRAP-0 - ... about a page of these tests - + TEST-TS1-5 - all 78 passed - T - * (use-package :tm) - T - * (≠ 1 0) - T - * - - -

Notice I used git tag to see the releases. At this time, v0.7-alpha is the latest, so I checked that out. If you want the unstable latest code rather than the stable latest release, leave out the git checkout instruction.

- -

The TM Library makes use of Unicode. There is no getting around it. This is discussed further in the next section. In addition TM defines synonyms for instructions such as 'not equal', which is the one instruction shown at the end of the transcript given above.

- -

Unicode usage

- -

For your convenience there is a file "emacs-keys" in the docs directory of the distribution. It sets the C-x g name SPC instruction to enter one of the Unicode characters that are used in the library. Here 'name' is a nickname.

- -

So to type the character capital delta after emacs-keys has been loaded, type C-x g D SPC. Actually Δ occurs twice in Unicode, once as capital delta, and once as a symbol for 'increment' in mathematics. We consider the increment version to only be there for typography purposes. We only use capital delta, even when it is for an increment variable.

- -

I've limited the use of Unicode mostly to things that 'probably would have been this way had Unicode been around before'. This includes conventional notation and a couple of symbol extensions that were needed to facilitate the TM access language.

- -

In the file src-0/fundamental.lisp find synonym bindings for the usual operators and common symbols such as ∧, ∨, ≥, ≤, λ, ∅, etc.

- -

Specific to the library we use the character '◧' as shorthand for 'leftmost'. This is because it looks like a little tape with the inked over cell being the leftmost cell. In the same manner the character '◨' is shorthand for rightmost.

- -

We use '➜' in continuation function names, and '⟳' as a loop operator.

- -

The ◧ and ◨ symbols are used in compound instruction names and for access language statements to indicate operation on rightmost or leftmost rather than the cell the head is on, or, to specify contracts with the programmer of the sort: "this function is only called when the cell is on rightmost."

- -

Since I had symbols for leftmost and rightmost, I started using them generally to mean leftmost or rightmost wherever it was convenient. For example, to shorten up the names of continuations so that parameter lists would fit on a line.

- -

Synonyms

- - - (defmacro defsynonym (old-name new-name) - "Define OLD-NAME to be equivalent to NEW-NAME." - `(defmacro ,new-name (&rest args) `(,',old-name ,@args)) - ) - (defconstant ∅ nil) - (defsynonym /= ≠) - (defsynonym <= ≤) - (defsynonym >= ≥) - (defsynonym not ¬) - (defsynonym and ∧) - (defsynonym or ∨) - (defsynonym string/= string≠) - (defsynonym string<= string≤) - (defsynonym string>= string≥) - (defsynonym lambda λ) - - -

Some reader macros

- -

q – a non literal quote

- -

In Lisp a quoted list is taken as being literal. However the result of modifying a literal is undefined, and often leads to bad results. Hence we provide the macro q which returns a quoted list which is not a literal.

- - - * (q a b c) - (A B C) - - -

{…} - unevaluated list

- -

When a form enclosed in parentheses, ( ... ), is evaluated the head is taken as the name of a function, looked up and called. The list members are also evaluated, and then passed as arguments to said function. If we don't want the head treated specially, but rather want to define a list, we can use a front item of #'list, which is the function to create a list.

- - - * (list 1 2 (+ 1 2)) - (1 2 3) - - -

We have defined a macro called L that like #'list, creates a list, but which also has some extra functionality.

- - - * (L 1 2 (+ 1 2)) - (1 2 3) - - -

We also provide a reader macro for L as braces.

- - - * {1 2 (+ 1 2)} - (1 2 3) - - -

If the apparent function open, #'o, appears inside of a call to L, then the arguments of the #'o function are included directly in the resulting list:

- - - * (defvar a {1 2 3}) - A - * (defvar b {4 5 6}) - B - * {a (o b)} - ((1 2 3) 4 5 6) - - -

Quoted non-literals can also occur within such an L list:

- - - * {a (o b) (q a b)} - ((1 2 3) 4 5 6 (A B)) - * {a (o b) (o (q a b))} - ((1 2 3) 4 5 6 A B) - - -

L acts identically to a quasiquote turned inside out. Whereas the default in quasiquote is to quote items, and a comma operator turns that off, the default in L is to evaluate items, and a q operator turns that off. Quasiquote has an @ marker to open up lists, while L has an o operator to open up lists. Inside of a quasiquote we could get in trouble if the name of a variable starts with an @ character; if such a variable appears after a comma, quasi quote will consider the variable name without the @ sign is to be opened and included. There is no analogous problem with the o operator because it only appears in the function channel. (The problem with quasiquote is that it uses 'in-band signaling' which mixes control with data in one channel.).

- -

[...] - head is variable holding a function name

- -

In Lisp the head of an evaluated list is taken as a function name. Consider this example that curries a two parameter function into a unary function by replacing one parameter with a constant argument of 3:

- - - (defun curry-three (f n) (f n 3)) ; has errors - - -

When we compile this function we get two errors:

- -

The variable F is defined but never used.

- -

and

- -

undefined function: F

- -

The first f is in the parameter list of the function definition, so it is taken as a variable name. In contrast the f in the body is at the head of an evaluated list, so it is taken as a function name. Hence there is a disconnect, and we get error messages describing this disconnect.

- -

The Lisp operator #' indicates that the symbol that follows is a function name to be taken literally, and not a variable name. This gets our function name into the data space for use as an argument. The Lisp function funcall is given as a first argument the name of a function to be called, while the remaining arguments are passed through to said function as its arguments:

- - - (defun curry-three (f n) (funcall f n 3)) - (defun plus (x y) (+ x y)) - (curry-three #'plus 2) - 5 - - -

In the definition for curry-three we do not get interpreter/compiler errors. This is because f is consistently used as a variable name. funcall will use the value of the variable f as the name of a function to call.

- -

In the second line we define a function to pass into curry-three. I put something simple here for sake of discussion. plus is defined to be a function that takes two arguments and sums them.

- -

In the third line we use the #' operator to tell Lisp that plus is a function name to be used literally as a value. This will be a value passed into curry-three no differently than had we put a number or string instance as an argument. Inside of curry-three the #'plus becomes the value of the variable f. Then the funcall function will use this value as a function name, and then call it.

- -

This is how function pointers are handled in Lisp as it does not have an explicit pointer type.

- -

Actually we didn't need to define the function plus, because '+' is already a function. We don't have reserved operator symbols in Lisp, instead we have loose rules on what can be used for function names.

- - - (curry-three #'+ 2) - 5 - - -

We introduce a shortcut with the TM Library. Normally a list to be evaluated is in parenthesis, and its head is taken literally as a function name. With the TM Library loaded, when a list in square brackets is evaluated, the head is taken as a variable name, and the value of this variables is the function to be loaded. It is a nice coincidence that square brackets mean indirect addressing in many assembly languages.

- -

We implemented this feature with a reader macro which simply turns the square bracket list into a regular list and inserts the funcall as its head. This occurs before the Lisp evaluate phase sees the syntax.

+
+ Figure 12: In-address-order byte by byte copy +
Figure 12: In-address-order byte by byte copy
+
-

Using the square brackets we may define curry-three as:

+

In the second case, the identical data stream arrives with words serialized as bytes in little-endian order, but the receiving machine is big-endian. The system must reverse the bytes strictly on a word-by-word basis.

- - * (defun curry-three (f n) [f n 3]) - CURRY-THREE - * (curry-three #'plus 2) - 5 - +
+ Figure 13: Reverse order byte copy for words +
Figure 13: Reverse order byte copy for words
+
-

Here is another example. Suppose that instead of passing #'plus in as an argument, that we first assigned it to a variable and then pass the variable value as an argument:

+

When the data arrives, there is no way to know where the word boundaries are. Serialization destroys that structural boundary information. Without knowing where the words are, a system cannot determine when to perform the byte order reversal. Therefore, resolving this requires a different approach than the one used for bit order in bytes; the problem transfers into the software layer.

- - * (defvar our-fun #'plus) - * [our-fun 4 7] - 11 - * (curry-three our-fun 2) - 5 - +

Byte order within words becomes an application level design consideration, even though it possesses no structural relevance to applications.

-

Here the value assigned to the variable our-fun is a function name. The variable is then used as any other, and its value is passed in as an argument to curry-three. Note, all arguments are evaluated before the function is called, so variables are replaced with their values. And as we know, inside curry-three, funcall will take the function name value from the corresponding parameter and call it as a function.

+

In protocols like the Internet Protocol (IP), the specification explicitly defines the data offset for the packet header. Code executes stably and efficiently. However, IP lacks context regarding the word locations in the data payload it carries, so IP routes the payload up the abstraction stack, preserving all byte order conflicts.

-

Things can become a little confusing when the variable name has the same name as the function.

+

JSON provides a standard for expressing tagged structured data built from primitive types while using character-only data. The specification dictates how numeric character strings evaluate, allowing both little-endian and big-endian machines to safely exchange JSON character-encoded numeric data. However, converting all numbers to strings and back wastes computational overhead, and transmitting them over a channel consumes excess bandwidth. Both processes expend unnecessary energy.

- - * (defvar plus #'plus) - * (curry-three plus 2) - 5 - +

Various file formats and data communications standards serve specific classes of applications by explicitly defining where words that require reversal are located during machine-to-machine data transfers.

-

Here #'plus is the function name as data. plus when it appears as the head of an evaluated list is a function name, otherwise it is a variable name. It is little wonder that the Lisp dialect Scheme put function names and variables name in the same space. However by doing so they had to provide some automatic conversions between variable names and function names.

+

TTCA

-

Summary

+

The native integer data type for TTCA utilizes a high radix online number system. This number system functions as an extension of online arithmetic. Similar to online arithmetic, it employs serial most-significant-digit-first signed digit arithmetic. In contrast to standard signed arithmetic, the radix scales significantly higher, causing a digit to span at least a byte in length. The numbers are highly scalable, and the architecture mandates an analysis step at compile time to establish rigorous precision requirements and exact range bounds. This process is detailed in the next chapter. The architecture provides explicit conversion instructions to generate alternate number formats.

-

When evaluated:

- + Appendix: The Tension Between Formal Methods and Practical Architecture diff --git a/document/book/TM_v1.html b/document/book/TM_v1.html new file mode 100644 index 0000000..ebdc8e2 --- /dev/null +++ b/document/book/TM_v1.html @@ -0,0 +1,800 @@ + + + Multiple machines and sharing tapes + +

Suppose we unmount a tape from a halted Turing Machine, say machine T0, and then mount the tape on another Turing Machine as input, say machine T1. Suppose we do this so that machine T1 may calculate the length of the output made by the first machine. When we do this, we run into some problems.

+ +

Firstly, for a conventional Turing Machine, the tape that is mounted on T1 will be infinite, so no computational T1 machine will be able to process it unless there is a message on the tape telling T1 where the end of the input is. For our TTCA Machines, if T0 starts with a null tape, and then expands it, and T0 is computational, then at the time T0 halts, the tape will be finite. T1 can then use the continuation of stepping beyond rightmost to know that it has processed all of its input.

+ +

If a Turing Machine does nothing then halts, it will implement an identity relationship between input and output. If we do not want the input given to a machine to 'bleed through', we will have to erase it. A computational TTCA Machine starting from a null tape can only produce finite tapes, so T1 can erase whatever T0 writes to the tape. However, if we are analyzing, instead of running the TTCA Machine, we might discover that the tape length would be infinite if the machine could be run. We know that if we pass an infinite tape to a computational TTCA Machine, it would not be able to erase the tape. Fortunately we can derive this fact through analysis and proof without having to run the computational machine to see if it erased the tape.

+ +

Hence if we allow for infinite inputs, we should add an 'erase-to-end-of-tape' instruction to our machines, so that the computational machines may leave a tape with only their outputs on them. Our erase-to-end-of-tape instruction will be executed in a single step.

+ +

Once an input tape is mounted, T0 is a computational TTCA Machine given a finite tape, its output is a finite tape, and that is taken to be the input for T1.

+ +

T0 is required to provide meta information about the length of the active area. This approach is known as in-band signaling. Because length information is mixed with the data.

+ +

T0 updates a dedicated table where it keeps track of the location of data objects and their lengths. Such a table is called a symbol table, and such a system is called a type system. This is written to a separate tape, or it is inserted at a known location on the main tape. In the case it is written to the front of the tape, the data following will have to be moved when the table grows. In the case it is written at the end of the tape, the table will have to be moved when the data grows. In either case the symbol table entry lengths themselves must also be managed, typically they are either coded into the controller or make use of in band signaling. It is conceivable that a symbol table would describe itself.

+ +

We adopt a convention of maintaining a compact tape. As such we have no embedded empty-symbols in the active area. Then the empty-symbol marks the end of the active area.

+ +

Each machine has two tapes, an input tape, and an output tape.

+ +

A compact area is one that has only alphabet symbols (no empty-symbols). We can extend this concept to say that the density of an area is the ratio of alphabet symbols to empty-symbols.

+ +

The approach we use in modern computing is often the 3nd one listed above, that of the type system. We carefully account for the length of each instance of data. Then we build up each larger instance from smaller ones, and while doing so, we add the lengths of the smaller instances to calculate the length of the larger instance. All programs then specify when they make or compose instances and their types.

+ +

A system for creating the appearance of many tapes over a single tape is called a memory manager or a memory allocation system. It is typically better to use one of these and multiple tape models rather than solving the problem of moving objects around in memory in an ad hoc manner.

+ +

Here is an important question for the TTCA Machine model: is it possible to define a memory manager that creates the appearance of multiple expanding tapes over the top of a single expandable tape? This problem is solved for managing files on a hard disk. However, all current file systems break at some point when pumped. So the question being asked here is equivalent to asking if a files system can be made without such architectural limits. The answer to this is yes, it is possible, as we will see in a later chapter of this book.

+ +

Hence there are multiple conventions we may use for implementing the abstraction of multiple tapes on a tape machine. Multiple tape machine are equivalent in power to one tape machines, but sometimes they are easier to think about.

+ +

Now as we have multiple tapes it is fairly easy to show that having multiple tape machines is equivalent to having one machine. We first view the multiple tapes worked on by the multiple machines as the multiple tapes of one machine. We then compose the state controllers in series, where the halt state of one is the initial state of the next one. Thus we may also conclude that having multiple tape machines is not more expressive than having one tape machine. We may also look at our multiple composed state controllers as one multiplexed controller, and then conclude that having multiple machines will not have an order of speed or space usage advantage.

+ +

The advantage of having multiple tape machines each perhaps having multiple tapes is that sometimes it is easier to think about. It is a method of partitioning the problem.

+ +

Suppose our machines have two tapes where one is called the input tape, and the other tape the output tape. Our procedure for passing tapes between machines will consist of umounting the tape from the source machine, then taking that tape over to the destination machine and mounting it as the input tape.

+ + +

Abstract areas and partitions

+ +

+ A tape area and partitioning can be an abstraction defined by a function rather than merely by a leftmost and rightmost cell. Such areas can have different topologies than those of the base tape. A familiar example for most computer scientists is utilizing software to create the appearance of a two dimensional array over a linear memory. +

+ +

+ Accordingly, suppose there is a three tape Universal Turing Machine gasket machine that holds the definition of a base machine on a first tape. It calls the base machine as a subroutine to access the base machine's tape (the second tape), and it uses its own tape to organize the tape abstraction. Then this outer machine can present to its user a variety of transforms of the base machine's tape. +

+ +

+ As an example, a gasket machine could partition the base machine's tape into two areas, one consisting of the odd addressed cells, and the other of the even addressed cells. Though these two areas consist of noncontiguous cells on the base tape, when viewed through the gasket machine, they appear perfectly contiguous. In this case, each area will have a leftmost cell and remain open on the right. +

+ +

+ When the base machine tape cells behind an abstract area are physically contiguous, we say that the abstract area is compact. In the odd even tape partition example, the abstract areas lack compactness. +

+ + + Zero length is a second order concept + +

Suppose we have a Turing Machine that is designed to mark an area based on some property of the symbols.

+ +

Suppose further that starting with the head on the first cell, our Turing Machine will step right zero or more times until it finds a cell that holds a symbol that has the special property.

+ +

Once it finds such a cell it will write an area marker symbol to that cell, step right, and repeat writing area marker cells until it finds a cell that holds a symbol that lacks the area property. At which point the machine halts.

+ +

Once an area is marked, we can go back and run a length measuring machine that counts the sequence of marks.

+ +

However we have a couple of problems, if there is no cell on the tape that has the special property. The area marking machine will step right without halting while looking for the marker. As a second problem, if the all the cells to right of the leftmost area cell, the area marker machine will never halt. +

+ + + +

Now suppose we employ a second order analysis.

+ +

Instead of running our area marker Turing Machine, we examine its definition and the definition of the machine that generated the tape data, to learn if such a machine will ever halt.

+ +

Although we know that it is not possible in general to analyze machines to know if they will halt, or not, it is certainly possible to do so in some cases, and this is one of those cases.

+ +

Upon analysis of our area marking machine we make a startling discovery: inputs exist for which the area marker will never halt.

+ +

In one case, if there is no leftmost symbol with the property that defines the area we are looking for, the marker machine will search forever.

+ +

In the second case, once the area is found, it is open on the right and it never ends.

+ +

We might say that inputs that have no leftmost symbol have an area of zero length.

+ +

This is an abstract concept, because our area marking and area measuring machines will never be run then halt and return a length measure of zero.

+ +

Instead we might arrive at this conclusion that a non-existent area has zero length through inductive reasoning: Say we have an area of length i, and then remove one element from the area, then it has length i-1.

+ +

As we repeat this, then eventually we will have an area of length 1 as discovered by our length assigning machine. Now we remove 1 more element.

+ +

Each time we removed an element before, it made the length smaller by 1, so we reason that 1-1 = 0. The area is now length zero.

+ +

We cannot go any further because there are no more elements to be removed.

+ +

In this case we arbitrarily imagined a longer area. No such longer area was stated in the problem formulation.

+ +

In real world programming, this is the difference between having a container that by implication will hold elements, and lacking the container entirely.

+ +

By applying the limit logic, we have implied that an area exists.

+ +

If running the area maker machine is asserting the existence of such an area, then it makes sense to speak of a zero length area.

+ +

However, if we are not making this assertion, then there is no area so the issue of length is irrelevant. This is a nuanced problem.

+ +

Note, then, that zero length is a second order concept. We cannot mark nor measure it, and given our first order definition for an area, nor can we even represent such an area at a first order.

+ +

There must be some external structure present that implies the existence of the area for zero length to even make sense.

+ +

This insight explains a lot of the pain related to the processing of end cases in computing. It will come up again in this volume when we examine the question of the emptiness of containers, see the section .

+ +

It also explains why loops so often need to be primed or given special case guards, which operates identically to adding a layer of analysis. We will discuss this further later when introducing the first-rest pattern see, .

+ +

It is interesting that non-existence has collided with zero length. This seems to be a contradiction, as something that does not exist should not have any length at all.

+ +

All of this happens at the second order, at the order of analysis. At this order we can make a distinction between an area that we have given a name to, and perhaps a location, as compared to an area for which we have done neither of these things.

+ +

Thus for purposes of second order analysis we will say that an area exists if it has a name or a location, even if it has zero length.

+ +

We will say an area does not exist if it has neither a name or a location. Again, execution of our first order area marking and length machines cannot provide us with any such information.

+ +

Need for the concept of cell

+ +

Let us ask a question, what is it that an address is actually locating? Let us consider this question in the light of an example. Suppose we have the tape sequence of:

+ + + a, b , c, ε, ε, ε … + + +

Now consider that we have an address of '2'. If we read tape address 2 we get back the letter 'c'. So the address is locating the 'c'. Now suppose we write at address 2. Say we write 'γ'. Now our sequence is:

+ + + a, b , γ, ε, ε, ε … + + +

Now we write 'Г', resulting in the sequence:

+ + + a, b , Г, ε, ε, ε … + + +

It would appear that the answer to our question is that address 2 is locating different things at different times. First it was, c, then γ, then Г. Yet the address did not change. It feels a little unsatisfactory to suggest that our concept of location depends on the value addressed. Also, notice, that when we made the Natural Number Turing Machine, that the values on the tape that was placed into correspondence with the Natural Numbers were never mentioned. Yet, we can't seem to answer the question of 'what is being addressed' without giving a value.

+ +

Addresses speak to the structure of the tape rather than the values held on the tape. So as to facilitate this interpretation, we note that a sequence consists of a sequence of cells holding elements, rather than being simply a sequence of elements. Given the concept of a cell we can say that an address always locates the same cell, though the contents of that cell can change.

+ +

This operates similarly to a small extension to the already existing concept of a variable in mathematics. In mathematics we allow that a variable can take on different values, though its name never changes. Now we are going to say that a cell can take on different values, though its address never changes. Furthermore, as the cell is part of the sequence, we are going to say the cell itself has a left neighbor or right neighbor, excluding the value in the cell.

+ +

Address of an area

+ +

The address of a cell is the number of steps required to reach the cell when starting from the leftmost cell on the tape. The leftmost cell has an address of zero. It might seem intuitive to set the address of an area on the tape to be that of the first cell in the area. If we require that an area have at least one cell to exist, this approach works even for machines with a cell delete instruction.

+ +

Suppose that we are deleting the cells in an area. The delete instruction affects the cell to the right of the cell the head is on. Hence to delete all the cells of an area, the head is placed on the left neighbor cell to the area. Say that we delete an area of three cells. We will call delete three times. It seems unsatisfactory to say the area no longer exists after the three deletes because the head locating the area never moved. I.e. we still have a location marker for the area, so we could, for example, call append and put a cell back into the area. For the area to truly not exist, it would not have a location.

+ +

Thus it seems more satisfactory to locate an area by its left neighbor cell, than to locate it by its leftmost cell.

+ +

The inverse case also suggests that the cell to the left of an area defines its address. I.e. if we call append to grow an area, it grows to the right of the cell the head is on. According to this definition of area location, the rightmost cell locates a currently zero length right tail of the tape that will potentially be grown.

+ +

In a sense what we are doing while exploring the meaning of area, existence, and zero length with these delete and append examples is performing a discrete limit in analogy to a limiting operation in calculous. To support having such a limit operation, the location of an area is that of the left neighbor of the leftmost cell in the area.

+ +

Not all is satisfactory. When using the left neighbor of an area as the location for an area, we cannot locate an area that starts on the leftmost cell of the tape. If we make this a special case, then we have failed to make a first order definition for area, because the special case information will have to be stored in a higher level structure that describes attributes of the tape.

+ +

Nor can this use the left cell approach work in the case of multiple areas.

+ +

Suppose we have two adjacent areas.

+ +
+ Tape with two areas +
Figure 2 Tape with two areas
+
+ +

Here we show a first area, say a0, that includes cells 7, 8 and 9. And a second area, say a1, that includes cells 10, 11, and 12. Thus, by the convention of using the address of the cell to the left of an area as the area’s address, a0 has an address of 6, while area a1 has an address of 9. Because the areas are adjacent, the address of area a1 is the same as the address of the rightmost cell in area a0.

+ +

We know that area a0 is located to the left of area a1 because a0's address is less than a1's. I.e. 6 < 9.

+ +

Suppose we delete cell 10. Though cell 10 is gone, our addresses remain consecutive, so what was cell 11, is now called cell 10, etc. and the diagram appears much as before, though area a1 is now only 2 in length:

+ +
+ Step by step area two becomes shorter +
Figure 3 Step by step area two becomes shorter
+
+ +

Now we delete the cell at address 10 two more times, and all the area a1 is gone. During the deletion, and just after, the head will be on the rightmost cell of area a0, i.e. on cell 9. We can now say we have an area of zero length located at cell 9.

+ +
+ Second area goes to zero length +
Figure 4 Second area goes to zero length
+
+ +

Now suppose after deleting the area a1, as just described, we continue on to delete area a0. Each time we delete the leftmost cell of a1 the rest of area a1 moves left by one. Hence, after the first deletion the address for empty area a1 becomes 8. Area a0 is still located to the left of a1, because 6 < 8:

+ +
+ Limit towards zero length being applied to the first area +
Figure 5 Limit towards zero length being applied to the first area
+
+ +

Finally when all cells in area a0 have been deleted, a1 has collapsed into a0. Both have the address of 6, so the order between them can no longer be derived from looking at the base addresses. Should we attempt to reverse the steps above, and only be given the machine at its final state we would have to begin by guessing what the order was between the two areas, a0 and a1.

+ +

So again, there is either missing information, or some information is not stored on the tape we are discussing.

+ +

As yet another problem case that is either not possible or requires external information, if an area includes the leftmost cell of the tape, then its location cannot be described with an address.

+ +

Hence this approach of using a discrete analogy to limits has led to some unsatisfactory end cases. We will visit this subject of areas on tape in the section, Area as a mapped tape and present a higher order approach for defining area location that avoids such end cases.

+ +
+ + +
+ + + Some properties of Turing Machines + +

For a given Turing Machine, the input is the sole determiner of the output. I.e. each time the same input is given, we get the same output. The input and output are mathematical objects, hence Turing Machines are functions. However, it is common in computer science to speak of Turing Machines as 'solving problems' rather than saying they are functions. This is because we often think of the inputs and outputs of Turing Machines as being something other than mathematical objects. For example, when a Turing Machine sorts sequences found on its input, we might say that it solves a sorting problem.

+ +

Turing Machines will differ due to differing alphabets, state controller graphs, associated instructions, start, and halt states. The choice of empty-symbol is inconsequential as long as it is distinct from the alphabet. As we will see in the later discussion on variations, the choice of alphabet is not very important. For two alphabets of the same cardinality we can setup a one to one mapping, and for those of differing cardinalities we can use sequences of alphabet symbols that map to alphabet symbols. For example, given an alphabet of ‘T’ and ‘F’, and a second alphabet of ’t’, ‘f’, ‘x’, ‘z’ we may make the following map:

+ + + f + | FF + + t + | FT + + x + | TF + + z + | FF + + +

Then given this mapping, we may use two cells for each one cell for any {‘f’, ‘t’, ‘x’, ‘z’} alphabet machine, and then use the only the {‘T’, ‘F’} alphabet.

+ +

There are a countably infinite number of permutations for alphabets, state controller graphs, associated instructions, start and halt states, hence there are a countably infinite number of Turing Machines that fit our definition. However, there are an uncountably infinite number of mathematical functions. Consequently, we must expect that some functions cannot be computed with Turing Machines.

+ +

There might be multiple Turing Machines that perform the same function. A set of such machines forms a functional equivalence class. Within a functional class there will be a class of members related in that they all use the smallest number of steps when considered against the limit of input length. We discuss this further in the section on complexity.

+ +

Of special importance to computation theory is the existence among all these infinite Turing Machines of a class of machines that read their state controller definition from the tape as an input. This is the Universal Turing Machine class.

+ +
+ + + Performance Analysis + +

An interesting aspect of the Turing Machine procedure is that it introduces the concept of stepping the machine. With the addition of some simple constraints it becomes possible to map the parts of the Turing Machine abstraction to the parts of some real machines. These constraints may take the form of such things as bounds on the length of the inputs, or the addition of out-of-resource errors. Because such constraints do not affect the 'normal' workings of the machine, the derived relationship between a Turing Machine step, and that of a unit of real time might not be that complicated. Indeed, except for some enumerable cases this relationship might even be so simple as to assign to a step an approximately constant amount of time.

+ +

Because of the existence of a relationship between steps and time, particularly when it is a simple one, it is very interesting for us to know how many steps a Turing Machine will take.

+ +

There are many ways to measure the complexity of a Turing Machine. Among these is something called the time complexity, which is a function that relates the length of the input to the number of steps required to reach the halt state.

+ +

To derive time complexity we typically start with a step count formula which maps the length of Turing Machine input to the worst case largest number of steps. We then consider the behavior of this formula as input length goes towards infinity. To get this, we take the highest order term from the step count formula. Conventional results are constant time, polynomial time, or exponential time.

+ +

We can derive the 'worst case length of the area written or read by the machine during computation' function in an analogous manner as for the step count function. This function is known as the space complexity. We may also consider the limiting behavior of this function to derive an order of space complexity.

+ +

The order of time or space complexity will remain the same against certain variations of our Turing Machine definition. For example, if we double all the states by adding a second state that we always visit, where this second state does nothing, the number of steps would double but the functionality would not change. Order of complexity also would not change. A fixed time machine before doubling up on the states would still be a fixed time machine afterward. It is simply that the number of steps would be twice as large, but still a fixed number relative to the size of the input. A polynomial time machine would still be polynomial time, simply with double size constants. We say that changes which do not change order of complexity, nor existence proofs, are inconsequential.

+ +

Suppose we have a complete Turing Machine functionality class. We say that it is complete because all possible machines for implementing the function are in this class. Some machines in this class will have a different order of time complexity than others. Now we consider the set of minimum order of time complexity machines from this class. As the larger set was complete, the set built against this constraint will also be complete relative to the constraint. We then say that this minimum order of time complexity is a property of the problem being solved, rather than being a property of a particular machine.

+ +
+ + + Conventional Turing Machine variations + +

In the first section of this chapter we gave a rather conventional definition for a Turing Machine. In the prior section, Complexity, we noted that we can analyze Turing Machines to find their time and space complexities. In this section we will discuss some variations that a person finds in the literature.

+ +

A variation on the conventional Turing Machine definition is allowed when it can be proven that the variation never causes existence, order of time complexity, nor order of space complexity results to change, and in this respect is inconsequential. Earlier we gave the example of doubling up the states as being such a variation, though that is an excluded variation typically.

+ +

Open in both directions tapes

+ +

Some Turing Machine descriptions describe a tape with no end in either the left or right directions, i.e. a tape that is open in both directions.

+ +

This feature adds no richness of expression, because we can get the same behavior from a Turing Machine with a single ended tape. To do this we partition the single ended tape into odd addressed cells and even addressed cells. The odd cells are said to be the right side of the tape, and the even ones the left side. We then rewrite any tape controller based on a bidirectional tape to instead use the 'odd' and 'even' channels instead of the left and right sides of the tape.

+ +

This same approach form can be used to show that multiple tapes, or even multi-dimensional tapes, add no expressive power. The good news is that such variations can be used whenever convenient, and we will get the same results.

+ +

Going in the other direction, the open in both directions tape is not a simplification. There is still a start cell, being the cell that the head is initially placed on. And as noted above, the topology around this start cell is no different, it is simply a question of the adjectives we use for describing it.

+ +

Alphabet replacement

+ +

Without loss of generality, we may replace the alphabet with a single symbol, say 's' (short for successor). This is because symbols in any alphabet can be placed into correspondence with a sequences of 's' symbols. For example, the symbols of the alphabet of {w, x, y, z} can be placed into one to one correspondence with the sequences in the manner of {<w, s>, <x, ss>, <y, sss>, <z, ssss>}. We will need to put the empty-symbol between any such sequences on the tape, so that two sequences can be distinguished from one longer sequence.

+ +

Alternatively we can adopt a two symbol alphabet instead of a single symbol, where the second alphabet symbol is an end of sequence marker.

+ +

In contemporary computing we use an alphabet of two symbols, {0, 1}, and fixed length sequences. No end marker is needed when the sequences to be placed into correspondence are of fixed length. So for example, the symbols in the alphabet {dog, cat, mouse, fish} may have the correspondence of {<dog, 00>, <cat, 01>, <mouse, 10>, <fish 11>}. Conventional fixed sequence lengths are 8, 16, 32, and 64. So for example, when the sequence length is 8, any alphabet of 256 symbols or less may be placed into correspondence. A conventional correspondence table is that of the ASCII code.

+ +

The fixed length to be used can depend on computational context. (In contrast, UTF8 does not use fixed length sequences, so there must exist at least one end of sequence marker.)

+ +

Another alternative to end of sequence markers for variable length sequences is to externally account for sequences lengths. We call such an accounting system a type system.

+ +

Leaving out the empty-symbol

+ +

When we use a fixed sequence of {1, 0} to stand for symbols, it is expensive to reserve a sequence for the empty-symbol. This expense is due both to losing the use of a symbol in the alphabet, and in the complexity of control circuitry when keeping track of it.

+ +

The empty-symbol functions as a property of the machine rather than simply another alphabet member, because the tape initially has an infinite tail of empty-symbols. A computational Turing Machine is limited to taking a finite number of steps. Thus it cannot compute a tape initialized with an empty-symbol (or any other value).

+ +

However we can add a constraint on all Turing Machine controllers that requires of controllers that they never write the empty-symbol, and always write an alphabet symbol to a cell before reading it. Then, because a cell is never read before being written, it does not matter what we write into it for initialization. We may even use an alphabet symbol. Consequently this constraint allows us to eliminate the empty-symbol. This gives us the following Turing Machine variation:

+ + + 1. fixed finite alphabet + 2. fixed write before read constraint + 3. variable single ended tape + 4. variable read/write head + 5. variable read value buffer + 6. fixed controlling state machine + 7. fixed left from leftmost error + 8. fixed start state + 9. fixed halt state + 10. variable current state + 11. fixed procedure for using these. + + +

To validate that this is an inconsequential Turing Machine variation, rather than a description of new abstraction that is not a Turing Machine, we must show two things: Firstly, that any of the now disallowed controllers never compute something that cannot be computed in the presence of the constraint. And secondly, that there are same complexity class alternatives for any disallowed controller.

+ +

No need to step into the inactive area

+ +

1. When attempting to step into the inactive area, instead keep a counter for the number of steps the machine would take. Only allow reads or writes or head movement when the counter is no longer needed due to the head having moved back into the active area.

+ +

2. Write an alphabet symbol and change the inactive area traversed into an active area.

+ +

The TTCA Machine variation

+ +

The active area on the tape can grow at most by one unit for each machine step. This largest growth occurs when the Turing Machine steps right and writes an alphabet symbol in every visited state. This means that for computational machines that start with a tape that has a finite input (active area), the output (active area) will be finite. This also means that space complexity can never be larger than time complexity.

+ +

A fixed value is one that is provided with a Turing Machine definition, and does not change while the machine runs. Suppose we chose a fixed length Turing Machine tape. The tape would then have a rightmost cell. That cell would have no right neighbor, but would have a left neighbor. We would also add another error, that of right from rightmost. This error would be invoked when the controller attempted to step right from the rightmost cell.

+ +

Consider a machine that does not step out of the active area, has constant space complexity, and where this space complexity is less than the fixed length for the finite tape; such a machine would never trip the right from rightmost error, and thus there would be no difference between a finite tape and an infinite one.

+ +

Now suppose that we bound the length of the input, and that the maximum space required for such inputs or shorter ones is less than or equal to the length of the tape. Then again, the right from rightmost error would never be taken, and thus the tape would be indistinguishable from an infinite one. (Today we typically pad programs with lots of memory and long address words in hopes this will be the case.) An analogous argument can be made if we bound the number of steps that may be taken.

+ +

Now consider the case where we do not fix the length of the input, nor the number of steps allowed, and that space complexity is such that space usage grows with growing input length, at least for very long inputs. For such machines we can always find an input of sufficient length to trip the right from rightmost error.

+ +

As another approach to finite computing we can run computations twice. For a given input we first run the Turing Machine variation that does not step out of active area but still has an infinite tape. We watch this machine closely while it is running and find the bound on the active area. Now we can make a second machine that has a fixed length tape at least as long as our active area measurement but is otherwise the same. Now with this second machine we can run the same input and there will be no right of rightmost error, and thus there will be no difference between having the finite tape or an infinite one.

+ +

Unlike for the constant space complexity proposal, and the bounded input length proposal, which only work for small subsets of potential inputs, this 'run twice' proposal derives a finite machine that works for any given input which a Turing Machine works for. Though, unfortunately, in all cases the second run will be moot, as we could have simply taken the output from the first run.

+ +

In a variation on the run it twice approach, instead of running the first machine, we might instead analyze it and should we be able to surmise a maximum tape length, we could use that.

+ +

In yet another approach we can extend the finite tape as needed. We place a control layer over the finite tape. When a step right instruction from the Turing controller invokes the right from rightmost error, the lower layer allocates memory, lengthens the tape, and then performs the requested step right. As long as this occurs in fixed time, (or of sufficient lesser order time than the dominate order of the time complexity), and as long as there is indeed more memory to allocate - this Turing Machine variation will yield the same order of computational complexity as one with an an infinite tape.

+ +

I propose the following Turing Machine variation:

+ + + fixed finite alphabet + write before read constraint + variable and extendable finite tape + variable read/write head + variable read value buffer + fixed controlling state machine + fixed left from leftmost error + fixed right from rightmost error + fixed start state + fixed halt state + variable current state + fixed procedure for using these + + +

In our original Turing Machine model, the controlling state machine instructions were limited to, do-nothing, left, right, write, with reading as an implied instruction. To this list we add append. The append instruction may only be called when the head is on the rightmost tape cell. This is not limiting because the instruction may be called from a state that is at the end of an arc triggered by the right from rightmost error. When we have no empty-symbol, append is given an alphabet symbol and performs a write into the new cell. This is not limiting because if need be, a person can always perform an extraneous write of an alphabet symbol.

+ +

With this extendable tape model all Turing Machine components remain finite during computation, though some are arbitrarily large. This variation is more suited for creating a mapping between a Turing Machine and a real program running on a real machine. Specifically, a computation requiring N cells maps exactly to a physical machine possessing at least N cells. The theoretical model and the physical implementation remain perfectly isomorphic provided the physical environment does not exhaust its memory bounds. Because a computational TTCA machine demands strictly finite tape extensions, a physical machine that completes the execution maintains exact structural correspondence. This isomorphism holds continuously as long as memory remains available, even accommodating dynamic hardware expansion such as memory hot-swapping. The theoretical correspondence fails solely upon a hard physical limit being reached during execution.

+ +

More about instructions

+ +

The Turing Machine state controller has a instruction symbol tied to each state. The Turing Machine procedure then has us take action based on this symbol. This is our current instruction set:

+ + + right, causes the head to move to the right neighbor cell. + left, causes the head to move to the left neighbor cell. + read, returns the symbol under the head. The returned symbol is then used to chose the transition arc. + write(x), causes the symbol x to be written to the cell that the head is currently on. The symbol x is any symbol from the alphabet. + append(x) may only be called when on the rightmost cell. Extends the tape by one cell, and writes the symbol x into that cell, where x is any symbol from the alphabet. + + +

We are going to relieve the constraint that append can only occur from rightmost. Our new append is functionally identical to adding a cell to the rightmost extremity, and then shifting all the symbols over by one cell starting at the new cell and ending when the new rightmost has been written - and then doing the requested write of x on the right neighbor cell.

+ +

We will also include the inverse function for append. delete(append(x)) reads x while deleting the cell that x was in. The current Turing Machine model can emulate this function by shifting all the symbols in cells the right of the head left by one, and then simply not using the rightmost cell.

+ +

We are also going to support multiplexed state controllers. Our multiplexed Turing Machine will have multiple heads. One for each separate thread of execution through the state controller. As explained in the following chapters, supporting multiplexing makes our machine more complex, especially in the presence of the delete instruction. However, we defer that discussion to the relevant chapters.

+ +

In cases where successive states are visited in a fixed order it is convenient to combine the instructions. We have developed the concept of a instruction statement to support this:

+ + + statement::[direction]instruction+[modifier][&contract]*[arg]* + direction::- | ε + instruction::r | w | s | a | d | m | e | ☥ + modifier::◧ ◨ n + instruction + r read cell under the head + w write cell under the head + s step + a allocate/add/append a new cell + d deallocate/drop/delete cell + m move, no allocation or deallocation of cells, requires fill + e entangled copy + ☥ entangled copy on a new thread + modifier + ◧ operate on leftmost + ◨ operate on rightmost + n repeats n times, n provided through an argument + contracts + h◧ head is at leftmost + h◨ head is at rightmost + examples + -s ; step left + a◧ ; make a new leftmost cell + a◨ ; append to rightmost + sn ; step n times + to derive a longer instruction, combine them: + as ; append then step + -a-s ; append to the left, step to the left + + +

The left direction is specified with a minus sign, otherwise the direction is taken as right going. So the letter s is the right instruction, and -s is the left instruction. The instruction s3 steps right three times.

+ +

The instruction a appends and writes a new cell to the right of the head. We use two special characters from the UTF character set to signify the rightmost and leftmost of the tape. This one looks like a little tape with its left cell inked in, ◧, so we use it to stand for leftmost. We use this one, ◨, to mean rightmost. Hence a◨ makes a new rightmost cell, and a◧ makes a new leftmost cell.

+ +

In some cases it is possible to implement higher performance implementations for instructions when the programmer tells us some additional information. For example a◨&h◨ has identical functionality as a◨, while the programmer also guarantees that we are on the rightmost cell. This saves the function from having to scan to the end of the tape.

+ +

We can concatenate the instruction letters into a string to summarize what would happen sequentially in adjacent state transitions. If these compound instructions need arguments, then they are pulled from the argument list in order as they are needed. For example, as means to append, with the parameter for the append taken from the argument list, and then to step.

+ +

We support multiplexing with the instruction e, which is short for entangled. This operator returns what appears to be a second independent machine, but this apparently independent second machine actually shares the same tape with the first machine. It is functionally identical to giving one machine two heads, and thus the ability to have state sets.

+ +

The instruction esr is a compound instruction referring to sequentially applying three other instructions. The e says to make an entangled copy of the head. The s says to step this copy, and the r says to do the read. The analogous esw does a write as the last step. This sort of combination of letters to make more complex instructions was inspired by Lisp's car and cdr compositions. Though this is functionally what the instruction does, its actual implementation might be completely different.

+ +

The combination of multiplexing and cell deletion leads to a hazard where one thread can delete a cell the head is on in another thread. We add a collision error continuation to our multiplexed Turing Machine interface because of this situation.

+ +

Chapter discussion

+ +

The modifications to the Turing Machine leading to our TTCA machine were inconsequential, in that for order of complexity and existence proofs we may swap one machine for the other and the results will be the same.

+ +

The original Turing Machine had an infinite tape. In contrast the TTCA machine has a surprising property: for computational problems all of its components remain finite. This follows from the fact that during computation a machine makes a finite number of steps, so the tape can only be expanded to be a finite size.

+ + + Consequentiality across the design abstraction stack + + +

Choice of realization

+ +

+ Let us take the example of adding two Arabic representation numbers. Logically this is considered to be a logarithmic time problem. We break the operands into fixed length pieces, and adding them in pairs results in a carry per block. By recursively pairing the blocks and applying the carries, we generate wider carries. Thus we can show that in terms of the logic gates that must be traversed, the sum is a log time operation. +

+ +

+ Physics comes to a different conclusion. In the worst case, a carry into the least significant bit can affect the sum bit some physical distance away. As the operands get longer, this distance grows in proportion. So given the propagation of information at a fixed speed, the bounding evaluation time against growing operand width is linear time. Even if it is log time in gate count, at some point the interconnect delay will dominate. +

+ +

+ The logical analysis of the adder given above allowed for unbounded resources, because as the adder operand increases in size, the number of block adders increases without bounds. In any realization there will be a limit on the number of blocks that can be added in parallel. These groups are then processed one by one, and the carry is propagated between them. Consequently, as the operands grow in length without bounds, the adder evaluation time becomes proportional to the number of groups processed. Processing groups in series is a linear time algorithm. +

+ +

+ It is notable that the time-multiplexed use of computer resources produces the same linear time result as the physics of information propagation analysis for the adder. +

+ +

+ A Turing Machine program faces a situation analogous to physical constraints. Given the operands are found on the tape, and the carry-in can affect the msb of the sum, the head will have to move ever more cells rightward to convey that lsb information up to the msb. Based solely on the propagation time of that information, addition is found to be a linear time algorithm. This propagation remains computation class limiting even if the Turing Machine is given an unbounded number of independent heads. +

+ +

+ There appears to be alignment among physical limitations, resource limited computing, and steps spent by Turing Machines while they carry information across a linear tape. This alignment indicates that a reasonable realization will be computation theoretic inconsequential. +

+ +

+ At this point we have arrived at questions of the physics of computation. The above analysis made use of classical physics. Perhaps realizations based directly on principles from Quantum physics models will be computation theoretic consequential. +

+ + +

Choice of implementation

+ +

+ From Babbage's Analytical Engine of 1842 up to the transition to mechanical relays and vacuum tubes in the 1940s, calculating machines were implemented with gears. The basic principle is apparent to anyone who has seen a mechanical odometer. Consider adding numbers for example: given two odometers, step one back at the same time as stepping a second one forward; when the first one reaches zero, the second will hold the sum. This process can be optimized, but the general idea remains the same. For such machines, a step is a rotation of the main shaft. +

+ +

+ The Harvard Mark I machine had a main axle speed that maxed out at 3000 RPMs, say 2700 RPMs to keep our math simple. Then this is 2700 steps per minute. The ENIAC was a similar implementation, but one that called out the use of circular shift registers of vacuum tubes instead of mechanical gears. Because there were 10 tubes in a ring register, it took 10 clock ticks to complete one 'rotation'. The clock rate maxed out at 450 kilohertz. That would be one rotation every 1/45,000 of a second, or 2.7 million RPMs, an improvement of three orders of magnitude! +

+ +

+ Yet, the same program when run on the Mark I took the same number of steps as on the ENIAC. But more importantly, a linear time algorithm on the Mark I was still a linear time algorithm on the ENIAC, etc. Thus, these implementation differences were computation theoretic inconsequential. +

+ +

+ It feels unsatisfactory to leave out the tremendous difference in clock rates. So let us address this feeling by naming an ENIAC main shaft 'rotation' as a standard 'step'. If we do this, then a Mark I shaft rotation would be 1000 ENIAC steps. Yet, this would merely affect the linear constant in the step count formulas. The same programs can be run, with the same inputs, and asymptotic behavior is the same for both machines, because computation classes do not include the constants on the step count equation. Constant time remains constant; linear, polynomial, and exponential time classes are the same as before. Programs that decide questions would get the same answers when they completed. +

+ +

+ Because we made an ENIAC shift register turn completion a 'standard step', we have a relative measure, so there is something we can do to create a computation theoretic consequential difference. Suppose we have two ENIAC machines, and we send one speeding away from Earth at an exponentially increasing rate, i.e. increasing red shift, and we observe it from Earth. We will observe that the clock on the traveling ENIAC is growing ever slower, and that a linear time program running on it will be observed to have exponential time behavior. Unfortunately, relativity does not smile upon us, as the people on the spaceship would not see the inverse, a speeding computation on Earth, but rather they would also observe a slowing one. +

+ +

+ So then, instead we send a spaceship towards Earth, with increasing blue shift, and we would observe that spaceship's ENIAC getting faster and faster. This is still not a computation theoretic speedup, because it is not asymptotic. In finite time, said spaceship would run into Earth, or pass it by and then be red shifting. +

+ +

+ A designer could purposely slow the clock on a second ENIAC so as to emulate red shift. For this to be more than mere theater there would have to be physical reason to run a slower clock than necessary, for example perhaps for conserving an ever dwindling battery. But slowing computation down, or even stopping it, is typically not useful. However, going the other direction, an ever faster clock does not work, as there is a finite maximum physical clock speed. +

+ +

+ We get an increasing blue shift situation with Moore's law. If every generation transistors become exponentially smaller, and thus faster, and we consider step times in years, hopping from new realization to new realization, then indeed linear time algorithms on a single realization would be log time algorithms on the generational computer. But chances are this is not an asymptotic, i.e. limit to infinity, phenomenon either. +

+ +

Choice of organization

+ +

+ Superscalar and VLIW computers execute multiple instructions in parallel. Real data dependencies put limitations on how many instructions are available to be executed in parallel, but even discounting this, if a program were executed N instructions at a time, its time to execute would divide by N. This merely affects the linear component of the equation mapping step count to input length, and thus does not change the computation class. Superscalar and VLIW architectures do not affect decisions; indeed they are transparent to programs, so decider problem results cannot change. Hence these techniques are not computation theoretic consequential. +

+ +

+ In general, by definition, organizations do not change a program's view of the machine, as that is part of the architecture. So organizations will not affect decider results. Also, the memory operations will be the same, as that is viewable state, so space complexity does not change unless time complexity changes. +

+ +

+ The realization sets fixed resources, so any attempt at parallelization will be bounded, as in the superscalar and VLIW discussion above. Thus at best it can divide the execution time by N. +

+ +

+ Some organizations can arrange computation in a manner that the base clock can run faster than for other organizations. However, clocks run at a fixed maximum speed. On modern systems they can slow down to reduce heat dissipation or battery consumption, but that does not make programs faster. So if one organization has a faster clock than another, the ratio is merely a linear term contributor. Apart from stopping, there is nothing a clock can do to participate in the decision making of the program. +

+ +

+ Caching of values sent to the system memory again does not participate in the decision making of a program. We are at best looking at improvements in the linear term. +

+ +

+ Branch prediction saves the time required to do a full fetch, but fundamentally it does not change the data flow graph of the program. The same decisions are made. +

+ +

+ Suppose that an organization keeps the operands for a function in a content-addressable memory. When the operands are recognized, it then immediately returns the looked-up value. This approach, called memoization, bypasses the internal looping of the function. Hence, this does participate in the decision making of a program, and could potentially change the computation theoretic complexity of programs with certain properties where the same operands occur in patterns and lead to an expensive computation. Memoization is typically designed into computer languages, rather than being built into the organization of a computer. Chances are, at the computer organization level or lower, it is more efficient to simply execute the presented instructions. +

+ +

Choice of architecture

+ +

+ Common decisions made at the architecture level are those for supporting RISC or CISC, the bit layout and handling of operands, the size of the internal register file, how DMA is to be handled, whether to use memory-mapped I/O or have explicit instructions for it, how interrupts are to be implemented and the number of entries in the interrupt table, what special registers are present and what features are available through them, how virtual memory and its user and process IDs are to be implemented with the possible use of a translation lookaside buffer, what onboard execution units will have direct instructions, the built-in data types, questions of unaligned accesses, bus standards to be supported, if sleep modes are to be present, how the machine will get booted, the security rings that will be supported, details of the hardware virtualization layer, special support for the OS, how the system stack will be handled, potential partitioning of address space, support for large buffers, and memory sharing features: none of these are computation theoretic consequential. +

+ +

+ As architecture enters the gray area with organization, cache architecture, bus layouts, bus buffers, direct inclusion of write buffers, perhaps a stack cache, prefetch buffers and split-transaction buses: none of these are computation theoretic consequential either. +

+ +

+ Said features certainly affect performance, but none participate in the decisions the program makes, change the number of execution steps by more than a linear ratio, or alter the memory complexity of the program. +

+ + + + +
+ + + TTCA Turing Machine in Lisp + +

Because our TTCA Turing Machine has finite sized components, we may create a software model for the TTCA Turing Machine without having to make assumptions of the sort that ‘very large approximates the infinite’. Rather we can show a one-to-one mapping of code and data in the software model and the TTCA variation of the Turing Machine. Consequently, the software model and theoretic model are isomorphic.

+ +

It follows that we can use our TTCA Turing Machine software to make theoretical statements about computation in the form of programs. By following this path we will learn some interesting things in this chapter about analysis, the meaning of emptiness, data type, the properties of non-destructive vs destructive programming styles, and multi-threaded programming among other things.

+ +

The Turing Machine, and our variation of it, may be partitioned into two parts. One part of the part consists of the tape head and the tape. We call this the Tape Transport Unit, as that is the name used for the mechanism that does this work on real tape storage units. The other part of the part is the Controller.

+ +

The Tape Transport Unit is given instructions for reading, writing, and moving the head. (In real Tape Transport Units the head is in a fixed position and we move the tape, but the relative affect is the same.) Our modified model adds instructions for extending the tape. During normal operation these instructions only come from the controller.

+ +

We have two types of controllers. One type of controller is a state machine. Its design is an integral part of the Turing Machine. To step the Turing Machine means to step this state machine to its next state. The state machine definition exists before the Turing Machine takes its first step, and its definition remains intact for as long as said Turing Machine exists.

+ +

When we speak of a Turing Machine without adding further words to the term as qualifiers, we mean that it uses this type of controller. For clarity we can call this a Directly Controlled Turing Machine.

+ +

The second type of controller is the Universal Controller, and a Turing Machine that uses this type of controller is called a Universal Turing Machine. The Universal Controller reads the definition of a state machine controller off of the tape. Hence the Universal Turing Machine may emulate any Directly Controlled Turing Machine. We can also call this an Indirectly Controlled Turing Machine. Here the qualifier ‘Indirectly’ is intended in the sense it is used in assembly languages, meaning the data is not present in the controller, but rather it must be fetched from memory.

+ +

With our Tape Machine library, your program is the controller, and the library implements the Tape Transport Unit. (In the future I will rename this library to ‘TTU’ from the current ‘TM’. Also the use of the term ‘function’ will be changed to ‘routine’.)

+ +

Your program that uses the TTU library is in a sense direct control, because the program is already defined before the first step of the machine, and with the possible exception of self-modifying code, it does not change until the program exits, and thus the process no longer exists. On the other hand, your program is loaded from memory by the processor, thus demonstrating that the processor is an indirect, i.e. universal, controller. Either interpretation works depending on how broad of a view we want to take.

+ +

Installing the library

+ +

TM is presented as an iteration library on the de facto package manager for Lisp quicklisp. Alternatively, a person may clone the repository www.github.com/Thomas-Walker-Lynch/tm and then checkout the latest release tag, which as of this writing is v0.7-alpha. After installing the code cd into the tm directory and run your lisp interpreter. Inside your lisp interpreter type the instructions (load “load”) and (test-all). test-all should return with a message that all of the tests passed.

+ +

It is possible that the threading tests, 'ts1-' might fail if your machine is heavily loaded or very slow, as they have timing built into them, but this is unlikely. Then type either (use-package :tm) or (in-package :tm) depending on what your objectives are.

+ +

The examples in this chapter either come from the tm/test directories, or from the tm/docs/examples directory. At the time they were placed in the book, they executed correctly, and I have endeavored to keep the examples up to date.

+ +

This is what it looks like when I follow install using the git clone method:

+ + + > git clone http://www.github.com/Thomas-Walker-Lynch/tm + Cloning into 'tm'... + remote:Counting objects: 3052, done. + remote:Total 3052(delta 0),reused 0(delta 0),pack-reused 3052 + Receiving objects: 100% (3052/3052),2.41 MiB|1.52 MiB/s,done. + Resolving deltas: 100% (2366/2366),done. + > cd tm + > git tag + v0.1-alpha + v0.7-alpha + > git checkout v0.7-alpha + Note: checking out 'v0.7-alpha'. + You are in 'detached HEAD' state ... + > sbcl + This is SBCL 1.3.14.debian ... + * (load "load") + ; compiling file "/home/tm/package-def/conditions.lisp" + ; compiling (IN-PACKAGE #:TM) ... about 10 pages of these + hooking test: TEST-TS1-5 + T + * (test-all) + + TEST-UNWRAP-0 + ... about a page of these tests + + TEST-TS1-5 + all 78 passed + T + * (use-package :tm) + T + * (≠ 1 0) + T + * + + +

Notice I used git tag to see the releases. At this time, v0.7-alpha is the latest, so I checked that out. If you want the unstable latest code rather than the stable latest release, leave out the git checkout instruction.

+ +

The TM Library makes use of Unicode. There is no getting around it. This is discussed further in the next section. In addition TM defines synonyms for instructions such as 'not equal', which is the one instruction shown at the end of the transcript given above.

+ +

Unicode usage

+ +

For your convenience there is a file "emacs-keys" in the docs directory of the distribution. It sets the C-x g name SPC instruction to enter one of the Unicode characters that are used in the library. Here 'name' is a nickname.

+ +

So to type the character capital delta after emacs-keys has been loaded, type C-x g D SPC. Actually Δ occurs twice in Unicode, once as capital delta, and once as a symbol for 'increment' in mathematics. We consider the increment version to only be there for typography purposes. We only use capital delta, even when it is for an increment variable.

+ +

I've limited the use of Unicode mostly to things that 'probably would have been this way had Unicode been around before'. This includes conventional notation and a couple of symbol extensions that were needed to facilitate the TM access language.

+ +

In the file src-0/fundamental.lisp find synonym bindings for the usual operators and common symbols such as ∧, ∨, ≥, ≤, λ, ∅, etc.

+ +

Specific to the library we use the character '◧' as shorthand for 'leftmost'. This is because it looks like a little tape with the inked over cell being the leftmost cell. In the same manner the character '◨' is shorthand for rightmost.

+ +

We use '➜' in continuation function names, and '⟳' as a loop operator.

+ +

The ◧ and ◨ symbols are used in compound instruction names and for access language statements to indicate operation on rightmost or leftmost rather than the cell the head is on, or, to specify contracts with the programmer of the sort: "this function is only called when the cell is on rightmost."

+ +

Since I had symbols for leftmost and rightmost, I started using them generally to mean leftmost or rightmost wherever it was convenient. For example, to shorten up the names of continuations so that parameter lists would fit on a line.

+ +

Synonyms

+ + + (defmacro defsynonym (old-name new-name) + "Define OLD-NAME to be equivalent to NEW-NAME." + `(defmacro ,new-name (&rest args) `(,',old-name ,@args)) + ) + (defconstant ∅ nil) + (defsynonym /= ≠) + (defsynonym <= ≤) + (defsynonym >= ≥) + (defsynonym not ¬) + (defsynonym and ∧) + (defsynonym or ∨) + (defsynonym string/= string≠) + (defsynonym string<= string≤) + (defsynonym string>= string≥) + (defsynonym lambda λ) + + +

Some reader macros

+ +

q – a non literal quote

+ +

In Lisp a quoted list is taken as being literal. However the result of modifying a literal is undefined, and often leads to bad results. Hence we provide the macro q which returns a quoted list which is not a literal.

+ + + * (q a b c) + (A B C) + + +

{…} - unevaluated list

+ +

When a form enclosed in parentheses, ( ... ), is evaluated the head is taken as the name of a function, looked up and called. The list members are also evaluated, and then passed as arguments to said function. If we don't want the head treated specially, but rather want to define a list, we can use a front item of #'list, which is the function to create a list.

+ + + * (list 1 2 (+ 1 2)) + (1 2 3) + + +

We have defined a macro called L that like #'list, creates a list, but which also has some extra functionality.

+ + + * (L 1 2 (+ 1 2)) + (1 2 3) + + +

We also provide a reader macro for L as braces.

+ + + * {1 2 (+ 1 2)} + (1 2 3) + + +

If the apparent function open, #'o, appears inside of a call to L, then the arguments of the #'o function are included directly in the resulting list:

+ + + * (defvar a {1 2 3}) + A + * (defvar b {4 5 6}) + B + * {a (o b)} + ((1 2 3) 4 5 6) + + +

Quoted non-literals can also occur within such an L list:

+ + + * {a (o b) (q a b)} + ((1 2 3) 4 5 6 (A B)) + * {a (o b) (o (q a b))} + ((1 2 3) 4 5 6 A B) + + +

L acts identically to a quasiquote turned inside out. Whereas the default in quasiquote is to quote items, and a comma operator turns that off, the default in L is to evaluate items, and a q operator turns that off. Quasiquote has an @ marker to open up lists, while L has an o operator to open up lists. Inside of a quasiquote we could get in trouble if the name of a variable starts with an @ character; if such a variable appears after a comma, quasi quote will consider the variable name without the @ sign is to be opened and included. There is no analogous problem with the o operator because it only appears in the function channel. (The problem with quasiquote is that it uses 'in-band signaling' which mixes control with data in one channel.).

+ +

[...] - head is variable holding a function name

+ +

In Lisp the head of an evaluated list is taken as a function name. Consider this example that curries a two parameter function into a unary function by replacing one parameter with a constant argument of 3:

+ + + (defun curry-three (f n) (f n 3)) ; has errors + + +

When we compile this function we get two errors:

+ +

The variable F is defined but never used.

+ +

and

+ +

undefined function: F

+ +

The first f is in the parameter list of the function definition, so it is taken as a variable name. In contrast the f in the body is at the head of an evaluated list, so it is taken as a function name. Hence there is a disconnect, and we get error messages describing this disconnect.

+ +

The Lisp operator #' indicates that the symbol that follows is a function name to be taken literally, and not a variable name. This gets our function name into the data space for use as an argument. The Lisp function funcall is given as a first argument the name of a function to be called, while the remaining arguments are passed through to said function as its arguments:

+ + + (defun curry-three (f n) (funcall f n 3)) + (defun plus (x y) (+ x y)) + (curry-three #'plus 2) + 5 + + +

In the definition for curry-three we do not get interpreter/compiler errors. This is because f is consistently used as a variable name. funcall will use the value of the variable f as the name of a function to call.

+ +

In the second line we define a function to pass into curry-three. I put something simple here for sake of discussion. plus is defined to be a function that takes two arguments and sums them.

+ +

In the third line we use the #' operator to tell Lisp that plus is a function name to be used literally as a value. This will be a value passed into curry-three no differently than had we put a number or string instance as an argument. Inside of curry-three the #'plus becomes the value of the variable f. Then the funcall function will use this value as a function name, and then call it.

+ +

This is how function pointers are handled in Lisp as it does not have an explicit pointer type.

+ +

Actually we didn't need to define the function plus, because '+' is already a function. We don't have reserved operator symbols in Lisp, instead we have loose rules on what can be used for function names.

+ + + (curry-three #'+ 2) + 5 + + +

We introduce a shortcut with the TM Library. Normally a list to be evaluated is in parenthesis, and its head is taken literally as a function name. With the TM Library loaded, when a list in square brackets is evaluated, the head is taken as a variable name, and the value of this variables is the function to be loaded. It is a nice coincidence that square brackets mean indirect addressing in many assembly languages.

+ +

We implemented this feature with a reader macro which simply turns the square bracket list into a regular list and inserts the funcall as its head. This occurs before the Lisp evaluate phase sees the syntax.

+ +

Using the square brackets we may define curry-three as:

+ + + * (defun curry-three (f n) [f n 3]) + CURRY-THREE + * (curry-three #'plus 2) + 5 + + +

Here is another example. Suppose that instead of passing #'plus in as an argument, that we first assigned it to a variable and then pass the variable value as an argument:

+ + + * (defvar our-fun #'plus) + * [our-fun 4 7] + 11 + * (curry-three our-fun 2) + 5 + + +

Here the value assigned to the variable our-fun is a function name. The variable is then used as any other, and its value is passed in as an argument to curry-three. Note, all arguments are evaluated before the function is called, so variables are replaced with their values. And as we know, inside curry-three, funcall will take the function name value from the corresponding parameter and call it as a function.

+ +

Things can become a little confusing when the variable name has the same name as the function.

+ + + * (defvar plus #'plus) + * (curry-three plus 2) + 5 + + +

Here #'plus is the function name as data. plus when it appears as the head of an evaluated list is a function name, otherwise it is a variable name. It is little wonder that the Lisp dialect Scheme put function names and variables name in the same space. However by doing so they had to provide some automatic conversions between variable names and function names.

+ +

Summary

+ +

When evaluated:

+ +