From: Thomas Walker Lynch Date: Mon, 10 Aug 2026 09:29:25 +0000 (+0000) Subject: wrestling with ftont matter X-Git-Url: https://git.reasoningtechnology.com/HU_reverse_machine.png?a=commitdiff_plain;h=dffeab418c301dc1113daaf5f93ea5a528d863e0;p=TM-2026 wrestling with ftont matter --- diff --git a/document/book/TM-2026.html b/document/book/TM-2026.html index 3e3a56b..b18e04b 100644 --- a/document/book/TM-2026.html +++ b/document/book/TM-2026.html @@ -32,23 +32,26 @@ - - - Acknowledgment + + Exordium -

The RT Iterator library work was sponsored by Reasoning Technology Inc. They have been kind enough to allow the library to be made public under an MIT license and placed on GitHub.

+

Experiencing the "Computer Arithmetic" community was indeed a Movable Feast.

-

This text was written separately, and was only possible due to the support of my loving wife Shihju.

- +

This book operates at the intersection of number theory, numerical analysis, and computation theory.

+ +

The RT Iterator Library work was sponsored by Reasoning Technology Inc. This was a first step in the creation of a computer language for increased computer security and behavioral introspection.

+

This book would not exist without the encouragement of my wife. While cousins brought their husbands to family events, she sat by herself as I remained home cavorting with Computation Theory. She is a beautiful, charming young woman. I am fortunate not to have grown horns on my head.

+ + - - Nomenclature + + Common Ground -

All communication is founded upon common knowledge, so here are some notes on the conventions used in this book.

+

All communication builds upon a foundation of common knowledge, so let us begin with the all too common discussion of terminology, type face, and other grounding.

-

In math spans and blocks, each container, such as a sequence or a set, is represented with a single capital letter, even when in Greek. Each object that is not a container is represented with a single lower case letter. Character pairs or full symbol names can also be used to represent math objects. In which case the same convention is followed for the first letter of the symbol, while the remain letters are lower case. The context will make it clear if a non-letter Unicode character represents a container or a non-container.

+

In math spans and blocks, each container, such as a sequence or a set, is represented with a single capital letter, even when in Greek. Each object that is not a container is represented with a single lower case letter. Character pairs or full symbol names can also be used to represent math objects, in which case the same convention is followed for the first letter of the symbol, while the remaining letters are lower case. The context will make it clear if a non-letter Unicode character represents a container or a non-container.

Code blocks and spans follow the RT code format conventions, which resemble the math conventions above but are not identical to them.For the complete specification, refer to RT-code-format.html and RT-code-format-Lisp.html located at https://github.com/Thomas-Walker-Lynch/RT-Style/tree/core-developer_branch/developer/document. For example, unlike for math scopes, comma-separated lists apply a space before the comma and bind the punctuation directly to the item being appended to the list:

@@ -66,11 +69,13 @@ if( f(g(x)) ){ + do_something(); + } Code . Padding applied to the outermost enclosure only -

Code containers are PascaleCase, and when allowed, non-container identifiers are snake-kebab_case, thus leveraging both hyphens and underscores to establish semantic binding precedence. An example of this is:

+

Code identifiers follow a parallel convention, though what counts as a container differs from the mathematical case. In code, a container is a namespace, a module, or a type, i.e. something that holds definitions, and these are written in PascalCase. Every other identifier is snake-kebab_case, and that includes the data structures that hold elements rather than definitions, so a list or a dictionary carries a lower case name. Both separators are in use so that a hyphen binds tighter than an underscore, which establishes semantic precedence. An example of this is:

@@ -92,11 +97,17 @@

Dates and times are given in the ISO 8601 format. The components run from the most-significant to the least-significant, year, then month, then day, in the same order the digits of a Hindu-Arabic numeral run. Two consequences follow from that ordering, and both are the reason for choosing it. A lexicographic comparison of two such dates agrees with their chronological comparison, so a list of them sorts correctly with no collation rules particular to dates. And the format is unambiguous, whereas a date such as 03/04/2026 is read as the third of April by some readers and the fourth of March by others. A time of day, when one is given, follows the date, and a trailing Z marks the time as UTC. Hence 2026-06-01 08:28:00Z. ISO 8601 itself separates the date from the time with a T, which is difficult to read. RFC 3339 anticipates the objection and permits a space in its place, and that is the option taken here.Graham Klyne and Chris Newman, Date and Time on the Internet: Timestamps, RFC 3339, July 2002, §5.6. Where a date appears without a time, the date alone is intended, e.g. 2026-06-01. As per the standard, if the day is not to be specified, it is simply dropped, e.g. 2026-06.

-

Several words in this book carry a formal definition that displaces their ordinary English sense, and those are capitalized: Peano Number, Integer, Rational, Real. The word real is the best example of why this is done. It was too much of an ask by mathematics to take a common English adjective and hold it to a technical meaning, and a reader is entitled to know which sense is in play. So a Real is the object mathematics defines, and a real problem or a real difficulty is the ordinary English word. Integer suffers no such ambiguity. It is capitalized for consistency with its siblings.

+

Several words in this book carry a formal definition that displaces their ordinary English sense, and those are capitalized: Peano Number, Integer, Rational, Real. The word real is the best example of why this is done. It was too much of an ask by mathematics to take a common English adjective and hold it to a technical meaning, and a reader is entitled to know which sense is in play. So a Real is the object mathematics defines, and a real problem or a real difficulty is the ordinary English word. Integer suffers no such ambiguity. It is capitalized for consistency with its siblings.

The leading capital letter on the mathematical number types does not distinguish the classical notion from the one constructed herein, because this book denies that the two are distinct in the first place. A Real is a Real whether a mathematician defines it or a machine produces the definition.

-

What mathematics calls the natural numbers are called Peano Numbers in this book. A Peano Number is what a Peano Machine outputs. This convention frees the word 'Natural' to refer to the philosophy only, so a reader never has to work out which of the two is meant. Note that herein Peano Numbers are taken to begin at zero, while Counting Numbers begin at one, and the two are not the same.Peano's own axioms began at one. Modern presentations of Peano arithmetic begin at zero, and that is the convention followed here. Giuseppe Peano, Arithmetices principia, nova methodo exposita (Turin: Bocca, 1889), §1. An Integer, a Rational, and a Real are likewise the outputs of machines, each constructed in its turn.

+

What mathematics calls the natural numbers are called Peano Numbers in this book. A Peano Number is what a Peano Machine outputs. This convention frees the word 'Natural' to refer to the philosophy only, so a reader never has to work out which of the two is meant. Note that herein Peano Numbers are taken to begin at zero, while Counting Numbers begin at one, and the two are not the same.Peano's own axioms began at one. Modern presentations of Peano arithmetic begin at zero, and that is the convention followed here. Giuseppe Peano, Arithmetices principia, nova methodo exposita (Turin: Bocca, 1889), §1. An Integer, a Rational, and a Real are likewise the outputs of machines, each constructed in its turn.

+ +

Number Theory consists of the analysis of Peano Number related structures. It gives properties to these structures, e.g. the familiar 'odd' and 'even'. It gives properties to the relationship between numbers and operations on those numbers, e.g. proving that an odd number squared is odd. It deals with the solution of mixed integer equations, and systems of equations, and sets the rules for modulus arithmetic. For example, encryption algorithms belong to Number Theory.

+ +

Numerical Analysis is built on two axioms. The first axiom limits computing to vectors of Peano Numbers, where a vector represents a number. A common representation is a number pair, where one number of the pair scales the other number, thus making it possible to represent fractional values. The second axiom states the actual intention of the programmer is to perform computation over a Real field. As Real Numbers fall on a continuum, it follows that some Real Numbers can not be represented with a vector of Peano Numbers. The objective of Numerical Analysis is then to establish the relationship between Peano Number results with the corresponding exact Real Number results. The difference between the two is called error. People who work in numerical algorithm design often spend a great deal of time trying to keep some metric of error low over a given domain.

+ +

Computation Theory consists of proofs over what can, and cannot be computed, the time and space complexity of algorithms, and the fundamental structure of computation. Number Theory is a kind of Computation Theory applied to Peano Structures before Church, Turing, and their contemporaries gave computation theory its name.

This book discusses four theoretical computing machines: two types of Turing Machine, which are not realizable, and two types of RT Machine, which are. What blocks realization of the first pair, and what removes the block from the second, is the subject of the chapters that follow.

@@ -112,33 +123,42 @@ Preface -

Bill Kahan had been instrumental in the specification of IEEE Std 754. Kahan had added some very insightful features. For example the representation switches to denormalized numbers so as to approach zero as closely as possible, with nothing wasted. Kahan also added a substitution flag for Infinity, which when the reciprocal was taken would produce zero. His stated reasoning was to approximate limiting arithmetic. What I saw in that was not limiting arithmetic per sé but rather an escape from a first order computation to something higher order. I wondered if there was not an analogy, perhaps a shadowing, with non-standard analysis.

- -

I went to Berkeley to talk to Bill about this, and we had lunch. We discussed errors and what do to with them. Bill was thinking in terms of interrupts and pipelines. I suggested a different path, that an error was merely an opportunity to do more computing. That the code design should be aware of the operands that caused errors, and thus they were not something to interrupt from, or to insert a symbolic value, but rather a branch, like any other. I was hoping that Bill would be interested in discussing it further.

+

The IEEE Std 754 floating-point standard equips the numerical analyst with the tools to design platform independent algorithms with predictable numerical behavior. Though the principal founder of the standard did commit one sin. He allowed for the expanded precision of the intermediate calculations on the Intel 8087 as a matter of backwards compatibility. He apparently succumbed to that final temptation because he had been part of that project. Consequently, the standard does not provide universal platform independence.

-

No long after Bill gave an interview with Dr. Dobb's Jack Woehr, "A Conversation with William Kahan," Dr. Dobb's Journal, November 1997, archived at https://jacobfilipp.com/DrDobbs/articles/DDJ/1997/9711/9711a/9711a.htm. The published piece opens with Kahan giving as his own thesis that exceptions are not errors unless they are handled badly, and that they are opportunities for extra computation. A related interview, "An Interview with the Old Man of Floating-Point," February 20, 1998, is sometimes cited alongside it. and there at the top was the exact quote that exceptions are opportunities for further processing.

+

One of the features of the standard is its support for a limiting arithmetic. If a computation divides by zero, an infinity flag can be substituted as a result. Then if the reciprocal is taken, zero returns. Zero and infinity can be signed, indicating the directions of the overflow and underflow. The substitution flags allow the program doing the computation to continue after what otherwise would be an interrupt exception. However, the intention is to give the numeric analyst tools, not to allow for programs that experience exception conditions to give right answers anyway. In fact, the error, i.e. the distance to the ideal result, most commonly increases rapidly after an exception indicating substitution enters the computation.

-

So of course I called Bill to ask about this. I had him on the spot, it was going to be an interesting conversation. Bill explained that the interviewer had edited the interview to make it dramatic, and that he was himself surprised upon reading it. I accepted that explanation, but I still wondered when was I going to get an opportunity to discuss my concepts? And I also know what it is like as I once had done a bad job discussing a Balgarian student's work at a conference in France, something I still feel embarrassed about when thinking about it.

+

Bill Kahan was said principal founder, and the standard happened because he is a staunch, sometimes overbearing, advocate of predictable numerical operations. In my experience in using the standard as a numerical analyst I was struck by how difficult it was to design accurate computations even when given predictable operations. It seems that another layer was needed for helping out the compiler, and I worked on this, and created an manual/automated process for the development of the transcendental functions by my numerics team at AMD. Then I arranged to meet Bill to discuss it.

-

John Gustafson also brought up the topic of the nature of error shortly before publishing The End of Error. He now has another proposal, the Posit, which abandons the variable width operands of his earlier unums in favour of a fixed width whose internal field boundaries move, so that accuracy is greatest for magnitudes near one and tapers toward the extremes of the range. This makes small numbers work even better than denorms did. John was polite and asked me about the method I had used to achieve accuracy in microcode. I answered with High Radix Online Arithmetic and with the importance of variable precision, but I could not formulate the rest of the thoughts. I had learned a great deal while doing formal analysis of the algorithms I had put on the Am29050, Am486, and K5 processors at AMD, yet when I tried to articulate this, I only stammered. So here passed a second opportunity to discuss the concepts of levels of analysis.

+

We had lunch in Berkeley, and I stressed the point that rather than stopping with operations and limiting arithmetic, that an exception during an operation was merely an excuse to do more computation. Clearly the concept resonated with Bill, as he stopped short after I said it. Also, not long after, Bill gave an interview to Dr. Dobb'sJack Woehr, "A Conversation with William Kahan," Dr. Dobb's Journal, November 1997, archived at https://jacobfilipp.com/DrDobbs/articles/DDJ/1997/9711/9711a/9711a.htm. A related interview, "An Interview with the Old Man of Floating-Point," February 20, 1998, is sometimes cited alongside it. and there at the top he was quoted repeating my words, "Exceptions are opportunities for extra computation."

-

Others have also addressed the problem of errors in computation. Moore originally introduce interval methods, where two numbers would bound a true value. Kulisch and Miranker attempted to formalize computer arithmetic, and started by reasoning about a screen that lay over the real number space, which is a route to the interval methods they embedded in their XSC languages. Jean-Michel Muller found a way to produce correctly rounded to full precision transcendental functions. Boehm and Cartwright introduced a concept of Constructive Real Numbers, where precision of computation was extended upon demand. And through Stephen Wolfram's vision, Mathematica provides a complete symbolic math manipulation package, where a symbolic expression can be reduced before being evaluated. Mathematica then tracks loss of precision through significance arithmetic. In my own work at AMD, employed an analysis step first, where required precision was derived, followed by variable precision computation.

+

Occasionally a student will come to me to share a great new discovery, that isn't so great, and isn't so new. Almost always the idea is not yet mature. Understandably a student's ideas are sometimes those of a person making observations while not realizing he is on a well trodden path. It is human nature to come to the conclusion that the unknown does not exist. This is not to say that professors do not harvest new ideas from students. My advice to such a student with such a discovery, is to first publish it, perhaps file a provisional patent, or at least write up and send it in an email rather than walking up and telling me about it, and then watch as the concept evolves over time and think what caused that evolution. Hence, the Lynch-Swartlander adder patent was filed in my name and that names of AMD design team. Also the original publication of High Radix Online Arithmetic at a small conference is in my name.

-

However, all of these methods are missing a key insight. That the further computation that the error invites us to engage in, is part of a richer second order system. The various systems mentioned above remain in the first order, though perhaps with a hint of the entering a second order analysis.

+

Credit and showing proper respect to both students and the hard work that got professors tenure is not as easy of a thing to do as it might seem. I was once asked to present a student's work at a conference in France, and was so anxious to talk about my own work so as to attract funding, that I nearly left the summary of hers out. I remain embarrassed over this to this day. Though I am sure her colleagues properly appreciate her work. Worse mistakes than this have been made. And what could I possibly have been thinking, a small conference on applied number theory in France is more than a world apart from a Silicon Valley tech meeting.

-

In practical computing the error algebra of IEEE Std 754 error can quickly mount due to the substitution flags, and the system tends towards the catch-all indeterminate value, the so-called NaN. This effect leaves some computer scientists wishing to extend the system. However this is not a failure of the system. The flags were intended to assist a numerical analysis expert in performing limiting arithmetic, not to make computations that have errors to work without being specially designed to do so. Extensions to the system do not usually fair much better, because error tag algebra can not serve as computation.

+

Of course I talked to Bill to ask about the quote. I calculated that I had gained a little leverage and thought to use it to perhaps get the support for the project. His backing would help the venture pitch. Bill was obviously quite sincere when he explained that the interviewer had edited the interview to make it dramatic, and that he was himself surprised upon reading it. And of all people to offer an explanation. He could easily have instead said that he had spent most of his career using exceptions as an excuse for more computation, and he would have been right about that. I gained some respect for Bill at that time, though alas, there was no project endorsement. That might be my fault, as come to think of it, I never directly asked for it. Furthermore, nothing in that Dr. Dobbs interview discussed the content, rather it was merely the sound bite enticing the next question: what is that further computation? This book discusses this topic.

-

Intervals will encompass error causing singularities that otherwise would not have manifested. However, the intervals tend to pessimistically widen and then pull in extraneous singularities. Exceptions can happen at run time without the code designer knowing of the potential for them. And when an except does occur, it tends to be handled ad hoc in the manner all exceptions are, for example by exiting the program. The interval system does not provide further guidance.

+

John Gustafson is another person who has put a lot of thought into how to formulate a valuable floating-point standard, and he now has a competing proposal. John also brought up the topic of the nature of error shortly before publishing The End of Error. He has improved and evolved that to the Posit, which abandons the variable width operands of his earlier unums in favor of a fixed width whose internal field boundaries move. This reminds me of the denorms of Std 754, but it is more graceful. Precision tapers away smoothly to the smallest representable value. John was polite and asked me about the method I had used to achieve accuracy in microcode. I answered describing High Radix Online Arithmetic and with the importance of variable precision, but I could not formulate the rest of the thoughts. I had learned a great deal while doing formal analysis of the algorithms I had put on the Am29050, Am486, and K5 processors at AMD, yet when I tried to articulate this, I could not explain it. It is articulated better in this book.

-

The Constructive Real is reminiscent of a limit, it will force the computation to provide sufficient precision. A limit concept is indeed key when moving from first order computation to a second order, but such an second order system is not built in. As for interval methods, an error could surprise an algorithm design when it appears at run time, and thus the code might not be prepared to handle it. However, there is an interesting aspect here, the expression syntax trees are available to the exception handler, or at least could be, so a layer could be built on top.

+

Many others have given thought to the problem of errors in computation and have made effective proposals. Moore originally introduced interval methods, where two numbers bound a true value. Kulisch and Miranker attempted to formalize computer arithmetic, and started by reasoning about a screen that lay over the real number space, which is a route to the interval methods they embedded in their XSC languages. Boehm and Cartwright introduced a concept of Constructive Real Numbers, where precision of computation was extended upon demand. And through Stephen Wolfram's vision, Mathematica provides a complete symbolic math manipulation package, where a symbolic expression can be simplified. Upon a call for numeric resolution Mathematica then tracks loss of accurate precision through significance arithmetic and repeats the computation at increased precision if the accuracy of the result is not sufficient. In my own work at AMD, my approach was to perform an analysis step first, where required precision was derived, then followed by variable precision computation.

-

Mathematica uses symbolic manipulation of expressions combined with significance arithmetic, so numeric resolution can fail, and then the user is left holding the error as a result. However, similarly to how the Constructive Real system has syntax trees, the exception handler can have access to the unresolved symbolic expression, which he could perform more computation with if he knew what to do with it. This is interesting.

+

However, all of these methods are missing a key insight. That the further computation that the exception invites us to engage in, is part of a richer second order system. The various systems mentioned above remain in the first order, though perhaps with a hint of entering a second order analysis.

-

There is a first edition of this book.TODO·cite: the first edition, and the Quicklisp release of the Lisp library. The same concepts are described in those pages, but the story is not as well told, and no one has come full circle to say that it made sense. One colleague who reviewed it admitted he was bewildered by it, and asked whether I was proposing that we build Turing Machines. After all, we are the music makers,Arthur O'Shaughnessy, "Ode", in Music and Moonlight (London: Chatto and Windus, 1874)., so to speak; and we write notes in transistors and layout. So then, he rightly asked, "What is the proposal here?" I have yet to give him an answer. Perhaps this second edition will help.

+

In practical computing the error algebra of IEEE Std 754 can quickly mount due to the substitution flags, and the system tends towards the catch-all indeterminate value, the so-called NaN. This effect leaves some computer scientists wishing to extend the system. However this is not a failure of the system as preventing this was not the design intent. Extensions to the system do not usually fare much better, because error tag algebra can not replace computation.

-

There is no hardware specification for a new proposed commercial machine in this book. This book is not proposing to build in hardware a Turing Machine. This is an absolute. Building a RT machine might be interesting. It is true that the organization for it is described in this book. Yet this is certainly not the proposal here. Rather this book explains and justifies a set of features that are planned to be incorporated into a commercial machine, one which I proposed to model and build in 2005. There was some interest, but ultimately we did not raise money for it.

+

With interval methods, the intervals will encompass error causing singularities that otherwise would not have manifested. However, the intervals also tend to pessimistically widen and then pull in extraneous singularities. Also, as for all these methods, exceptions can happen at run time without the code designer being aware of their nature or perhaps of the potential for it to happen. And when an exception does occur, it tends to be handled ad hoc in the manner as in conventional computation, for example by exiting the program as the interval system does not provide further guidance.

+ +

The Constructive Real is reminiscent of a limit, it will lazy evaluate for increased precision until obtaining a sufficiently accurate result. A limit concept is indeed key when moving from first order computation to a second order, but such a second order system is not built in. As for interval methods, an error could surprise an algorithm designer when it appears at run time, and thus the code might not be prepared to handle it. However, there is an interesting aspect here, the expression syntax trees are available to the exception handler, or at least could be, so a layer could be built on top.

+ +

Mathematica uses symbolic manipulation of expressions combined with significance arithmetic, so numeric resolution can fail, and then the user is left holding the error as a result. However, similarly to how the Constructive Real system has syntax trees, the exception handler can have access to the unresolved symbolic expression, which the user could perform more computation with if he knew what to do with it. This is interesting.

+ +

There is something more fundamental going on here that all of these approaches are dancing around. There are limits to what is possible with computing and even logic itself. Attaching number representations to the structures does not change this fact. Hence, this book will start not with numerical analysis, but with the foundation of mathematics and computation theory.

+ +

There is a first edition of this book.Thomas Walker Lynch, Tom's Turing Complete Architecture: Volume 1, Foundation and TM Library Interface (Independently published, April 24, 2023), 230 pp., ISBN 979-8392389087. The Lisp library described in that edition was released on Quicklisp. The same concepts are described in those pages, but the story is not as well told, and no one has come full circle to say that it made sense. One colleague who reviewed it admitted he was bewildered by it, and asked whether I was proposing that we build Turing Machines. After all, we are the music makers,Arthur O'Shaughnessy, "Ode", in Music and Moonlight (London: Chatto and Windus, 1874). so to speak; and we write notes in transistors and layout. So then, my colleague Kelvin rightly asked, "What is the proposal here?" I have yet to give him an answer. Perhaps this second edition will help.

+ +

So as to address Kelvin's question directly, and to prepare other readers who are familiar with my computer architecture work, let me say that there is no hardware specification for a new proposed commercial machine in this book. This book is not proposing to build a Turing Machine in hardware. This is an absolute. Building an RT Machine might be interesting. It is true that the organization for it is described in this book. Yet this is certainly not the proposal here. Rather this book explains and justifies a set of features that are planned to be incorporated into a commercial machine, and yes, that is the machine I was pitching for funding to develop in 2005.

+ +

Though a practical commercial processor design is not provided in these pages, there is a saving grace, i.e. a practical contribution. There is a formally grounded iterator library described and available for download from GitHub. This library appeared in the first edition implemented in Lisp (SBCL, Common Lisp, rather than Scheme) and released on Quicklisp. That library is described in an appendix. Lisp is in a sense pure and formal statements can be made with it, so writing the library in Lisp made sense. Certain improvements have been made since the first edition, and this volume introduces and discusses the Python version.

-

However, there is a practical aspect to this work. A software library that is of general usefulness can be found in these pages, and downloaded from GitHub. It was first articulated in the Lisp language in the first edition of the book and released on Quicklisp. It is described in an appendix. Lisp is in a sense pure, and formal statements can be made with it, so writing the library there kept it a small and natural step away from the formal discussion that precedes it. Since that time I have written parts of the library in other languages and found it useful. This volume develops the Python API, and the Python version is to be made available on Github.

 @@ -154,17 +174,19 @@

Here the Naturalist runs into a problem. The Turing Machine is not practical to build. The infinite tape is of course an issue, but it is perhaps not the most serious obstacle to realizing a Turing Machine. As I will formalize later in this book, the Turing Machine head must always read and react, and this will force the author of the controller to build memory into the control. In turn this causes the controller to explode in size with machine word width.

-

However, this situation is recoverable. This volume presents modifications that lead to an alternative computation theoretic object that is equally expressive as a Turing Machine, and yet is practical to build. This then restores the validity of the Natural interpretation.

+

However, this situation is recoverable. This volume presents modifications that lead to an alternative computation theoretic object that is as expressive as a Turing Machine, and yet is practical to build. This then restores the validity of the Natural interpretation.

For a question to be resolved by a Turing Machine, the question must be formed as a program that when run produces a result in a finite number of steps. Practicality is an even stronger constraint than being computational. For running a program to be practical, finishing in a finite number of steps is not sufficient; it must also complete before a given deadline. In the natural world, we all operate under the constraints of being practical. However, that does not prevent us from having a practical discussion about what is theoretically possible to compute, and such discussion is found in the pages of this book.

-

It is commonly thought that Turing Machines, and thus observations of nature, are limited to discrete computation, and thus there is a large domain of continuous mathematics that they can not speak about. However, the very concept of the continuous is described in discrete terms. Consider the simple function x^2 over a real field. It is considered to be a continuous function, yet it consists of discrete symbols. To take its derivative is to follow the steps of an algorithm, i.e. a program, which brings the 2 down as a multiplier and replaces the exponent with a 1, or leaves it out altogether. These are discrete steps resulting in another discrete formula, 2 x. In that the word continuous is given meaning, it becomes discrete.

+

It is commonly thought that Turing Machines, and thus observations of nature, are limited to discrete computation, and thus there is a large domain of continuous mathematics that they can not speak about. However, the very concept of the continuous is described in discrete terms. Consider the simple function x^2 over a Real field. It is considered to be a continuous function, yet it consists of discrete symbols. To take its derivative is to follow the steps of an algorithm, i.e. a program, which brings the 2 down as a multiplier and replaces the exponent with a 1, or leaves it out altogether. These are discrete steps resulting in another discrete formula, 2 x. In that the word continuous is given meaning, it becomes discrete.

-

Analysis is the bridge that leads from the natural world to the world of forms. Here analysis is the activity of asking about the properties of a program instead of running it. The program being analyzed then becomes the input data for the program doing the analysis. Turing's original paper introduced the naturalist interpretation of mathematics via the clerk at the desk, described a machine for automating the clerk's work, introduced the idea of a stored program, described analysis, and pointed out that it was possible to analyze an analyzer. And as though that was not enough, he provided a general analysis applicable to halting analyzers that proved that no universal halting analyzer can exist. (A halting analyzer is an analyzer, i.e. a program, that analyzes a given program to decide if that given program halts or not.) This does not mean that a halting analyzer can not ever be practical. For a large class of programs a halting analyzer can give an answer, and do so before a deadline. However, the proof result means that for any halting analyzer, there will be programs for which it will not be able to decide whether they halt or not, even when there is no deadline apart from 'finite number of steps'.

+

Analysis is the bridge that leads from the natural world to the world of forms. Here analysis is the activity of asking about the properties of a program instead of running it. The program being analyzed then becomes the input data for the program doing the analysis. Turing's original paper introduced the naturalist interpretation of mathematics via the clerk at the desk, described a machine for automating the clerk's work, presented the idea of a stored program, described analysis, and pointed out that it was possible to analyze an analyzer. And as though that was not enough, he provided a general analysis applicable to halting analyzers that proved that no universal halting analyzer can exist. (A halting analyzer is an analyzer, i.e. a program that analyzes a given program to decide if that given program halts or not.) This does not mean that a halting analyzer can not ever be practical. For a large class of programs a halting analyzer can give an answer, and do so before a deadline. However, the proof result means that for any halting analyzer, there will be programs for which it will not be able to decide whether they halt or not, even when there is no deadline apart from 'finite number of steps'.

Here then, is an interesting thing about some Turing Machine programs that do not halt. Though they can not be run to completion, they can be analyzed. Say a program prints the 's' character, then loops back, and thus prints the 's' character again and again. Such a program can not be run to completion as it never halts. However, the program can be analyzed and talked about, as of course we already know, as we are doing this very thing right now. Thus the holes in the lower level language kick the problem up to a higher level where questions are asked as to how the program behaves and what an exceptional condition means.

-

Accordingly the world of forms is not a separate realm. It is the tower of languages standing above the machine, each layer of it there to speak of the holes in the layer beneath. Those languages become separated from the machine that gave rise to them, so the base machine never need be run. At each moment, we reach the limit of what can happen in that moment, then a new moment arrives, both in stepping through programs, or in layers of abstraction.

+

Accordingly the world of forms is not a separate realm. It is the tower of languages standing above the machine, each layer of it there to speak of the holes in the layer beneath. Those languages become separated from the machine that gave rise to them, so the base machine never need be run. What builds the tower is exhaustion. Each layer runs out of what it can say, and the running out is what makes the next layer necessary.

+ +

The first exhaustion of this kind was not met in computing. It was met in mathematics, decades before there was a machine to describe it, and the Turing Machine is what the search for a higher view produced.

 @@ -174,7 +196,7 @@ The search that led to the Turing Machine

- In 1893 Gottlob Frege published an axiomatic construction of mathematics from set theory. Frege's grand objective was something he called Logicism, the philosophical thesis that all of mathematics can be derived entirely from pure logic. To bridge set theory and logic, Frege defined sets using a method known as set comprehension. Under this approach, a mathematician states a logical rule or property, and any object satisfying that logical statement automatically becomes a member of the set. Because the membership of a set is determined entirely by logical rules, the resulting sets, and the mathematics built upon them, are derived directly from logic. To implement this, his specific machinery relied upon unrestricted set comprehension, formalized as Basic Law V Gottlob Frege, Grundgesetze der Arithmetik, Vol. 1 (Jena: Hermann Pohle, 1893), §20.. + In 1893 Gottlob Frege published an axiomatic construction of mathematics from set theory. Frege's grand objective was the programme later called Logicism, the philosophical thesis that all of mathematics can be derived entirely from pure logic. To bridge set theory and logic, Frege defined sets using a method known as set comprehension. Under this approach, a mathematician states a logical rule or property, and any object satisfying that logical statement automatically becomes a member of the set. Because the membership of a set is determined entirely by logical rules, the resulting sets, and the mathematics built upon them, are derived directly from logic. To implement this, his specific machinery relied upon unrestricted set comprehension, formalized as Basic Law V Gottlob Frege, Grundgesetze der Arithmetik, Vol. 1 (Jena: Hermann Pohle, 1893), §20..

@@ -183,7 +205,7 @@

- In 1901 Bertrand Russell found a well-formed set formulation using Frege's set theory that did not correspond to a set. As Frege's work was based on this set theory, this called into question his entire work. Russell pointed out that it was possible to define a set of all sets that do not contain themselves. However this was a paradox, because if said set contained itself, it shouldn't, and if it didn't it should. Thus the formulation fails to define a set because the logical condition cannot be satisfied Bertrand Russell, The Principles of Mathematics (Cambridge: Cambridge University Press, 1903), Chapter X, 'The Contradiction'.. Russell communicated this to Frege in a letter dated 1902-06-16, shortly before Frege's second volume was going to print Bertrand Russell to Gottlob Frege, June 16, 1902, reprinted in Jean van Heijenoort, From Frege to Gödel: A Source Book in Mathematical Logic (Cambridge: Harvard University Press, 1967), 124–125. Gottlob Frege, Grundgesetze der Arithmetik, Vol. 2 (Jena: Hermann Pohle, 1903), Appendix (Nachwort), 253. Frege writes: 'Hardly anything more unfortunate can befall a scientific writer than to have one of the foundations of his edifice shaken after the work is finished.'. Frege hurriedly authored an appendix (the Nachwort) admitting his system was compromised Frege was a quiet, rigid man who had spent decades building his logical fortress in almost total academic obscurity. Frege was personally devastated by Russell's letter. Shortly after, he suffered the loss of his wife, fell into severe depression, and his academic output almost entirely ceased. In 1924, a year before his death, he wrote unpublished diaries explicitly surrendering his life's work, declaring that logicism was a mistake and that mathematics must actually be derived from geometry. Note I. Grattan-Guinness, The Search for Mathematical Roots, 1870–1940 (Princeton: Princeton University Press, 2000). For an analysis of Frege's intellectual decline, personal tragedies, and his unpublished 1924–1925 diaries where he formally surrenders the logicist program, see Chapter 7.. + In 1901 Bertrand Russell found a well-formed set formulation using Frege's set theory that did not correspond to a set. As Frege's work was based on this set theory, this called into question his entire work. Russell pointed out that it was possible to define a set of all sets that do not contain themselves. However this was a paradox, because if said set contained itself, it shouldn't, and if it didn't it should. Thus the formulation fails to define a set because the logical condition cannot be satisfied Bertrand Russell, The Principles of Mathematics (Cambridge: Cambridge University Press, 1903), Chapter X, 'The Contradiction'.. Russell communicated this to Frege in a letter dated 1902-06-16, shortly before Frege's second volume was going to print Bertrand Russell to Gottlob Frege, June 16, 1902, reprinted in Jean van Heijenoort, From Frege to Gödel: A Source Book in Mathematical Logic (Cambridge: Harvard University Press, 1967), 124–125. Gottlob Frege, Grundgesetze der Arithmetik, Vol. 2 (Jena: Hermann Pohle, 1903), Appendix (Nachwort), 253. Frege writes: 'Hardly anything more unfortunate can befall a scientific writer than to have one of the foundations of his edifice shaken after the work is finished.'. Frege hurriedly authored an appendix (the Nachwort) admitting his system was compromised Frege was a quiet, rigid man who had spent decades building his logical fortress in almost total academic obscurity. Frege was personally devastated by Russell's letter. Shortly after, he suffered the loss of his wife, fell into severe depression, and his academic output almost entirely ceased. In 1924, a year before his death, he wrote unpublished diaries explicitly surrendering his life's work, declaring that logicism was a mistake and that mathematics must actually be derived from geometry. See I. Grattan-Guinness, The Search for Mathematical Roots, 1870–1940 (Princeton: Princeton University Press, 2000). For an analysis of Frege's intellectual decline, personal tragedies, and his unpublished 1924–1925 diaries where he formally surrenders the logicist program, see Chapter 7..

@@ -207,12 +229,12 @@

- In 1928 David Hilbert and Wilhelm Ackermann published a textbook on mathematical logic, Grundzüge der theoretischen Logik David Hilbert and Wilhelm Ackermann, Grundzüge der theoretischen Logik (Berlin: Springer, 1928). This first edition has not been translated into English.. A feature of this book is its attention to procedures to follow for mechanically determining truth of statements. They called the problem solved by such a procedure the Entscheidungsproblem. In the first chapter they review the procedure for solving the Entscheidungsproblem in the propositional logic. For the first-order predicate calculus they define the problem as, "Universal validity concerns the following question: How can one determine, for any given logical expression that contains no individual signs [constants], whether the expression represents a true assertion for arbitrary substitutions for the occurring variables, or not?" Ibid., 72–73.. They review some special cases with solutions, including one published earlier by Ackermann, but then throw down the gauntlet by saying, - "A general solution to the Entscheidungsproblem, regardless of whether a person considers the first or the second formulation, is not yet available." Ibid., 81. "Eine allgemeine Lösung des Entscheidungsproblems, mag man nun die erste oder die zweite Fassung nehmen, liegt bis jetzt noch nicht vor." The term Entscheidungsproblem literally translates to 'decision problem'. However, there are many types of decision problems, and later we will meet a class of Turing Machine programs called deciders, so it appears to be best to keep the original German. As we will see later Alan Turing also did this.. + In 1928 David Hilbert and Wilhelm Ackermann published a textbook on mathematical logic, Grundzüge der theoretischen Logik David Hilbert and Wilhelm Ackermann, Grundzüge der theoretischen Logik (Berlin: Springer, 1928). This first edition has not been translated into English.. A feature of this book is its attention to procedures to follow for mechanically determining truth of statements. They called the problem solved by such a procedure the Entscheidungsproblem. In the first chapter they review the procedure for solving the Entscheidungsproblem in the propositional logic. For the first-order predicate calculus they define the problem as, "Universal validity concerns the following question: How can one determine, for any given logical expression that contains no individual signs [constants], whether the expression represents a true assertion for arbitrary substitutions for the occurring variables, or not?" Hilbert and Ackermann, Grundzüge der theoretischen Logik, 72–73.. They review some special cases with solutions, including one published earlier by Ackermann, but then throw down the gauntlet by saying, + "A general solution to the Entscheidungsproblem, regardless of whether a person considers the first or the second formulation, is not yet available." Hilbert and Ackermann, Grundzüge der theoretischen Logik, 81. "Eine allgemeine Lösung des Entscheidungsproblems, mag man nun die erste oder die zweite Fassung nehmen, liegt bis jetzt noch nicht vor." The term Entscheidungsproblem literally translates to 'decision problem'. However, there are many types of decision problems, and later we will meet a class of Turing Machine programs called deciders, so it appears to be best to keep the original German. As we will see later Alan Turing also did this..

- In 1931 Kurt Gödel published his incompleteness theorems Kurt Gödel, "Über formal unentscheidbare Sätze der Principia Mathematica und verwandter Systeme I," Monatshefte für Mathematik und Physik 38 (1931): 173–198.. By mapping formal logic into arithmetic, he demonstrated that any consistent formal system sufficiently powerful to perform basic arithmetic, let us call it system F, will inevitably contain well-formed formulas that are mathematically true yet cannot be proven within the system itself For the definitive English translation, see Jean van Heijenoort, From Frege to Gödel: A Source Book in Mathematical Logic, 1879–1931 (Cambridge: Harvard University Press, 1967), 596–616.. Gödel achieved this by engineering a specific formula that evaluates to the claim: "G: There exists no sequence of valid logical steps within system F that proves G." If system F is consistent, it cannot output a proof for G; thus, the claim G makes is factually accurate, rendering it true but mechanically unprovable. Furthermore, Gödel demonstrated that system F cannot output a proof of its own consistency. This result fractured David Hilbert's 1900 vision of utilizing a weaker, strictly "finitistic" logical subsystem to definitively prove that the axioms of arithmetic are entirely free of contradictions David Hilbert, "Mathematical Problems," Bulletin of the American Mathematical Society 8 (1902): 437–479.. If the full, powerful system F lacks the mechanical capacity to verify its own consistency, Hilbert's weaker finitistic subsystem is definitively incapable of accomplishing the task. Gödel's work established a hard mechanical boundary, asserting that truth and provability are distinct concepts in classical mathematics. + In 1931 Kurt Gödel published his incompleteness theorems Kurt Gödel, "Über formal unentscheidbare Sätze der Principia Mathematica und verwandter Systeme I," Monatshefte für Mathematik und Physik 38 (1931): 173–198.. By mapping formal logic into arithmetic, he demonstrated that any consistent, effectively axiomatized formal system sufficiently powerful to perform basic arithmetic, let us call it system F, will inevitably contain well-formed formulas that are mathematically true yet cannot be proven within the system itself For the definitive English translation, see Jean van Heijenoort, From Frege to Gödel: A Source Book in Mathematical Logic, 1879–1931 (Cambridge: Harvard University Press, 1967), 596–616.. Gödel achieved this by engineering a specific formula that evaluates to the claim: "G: There exists no sequence of valid logical steps within system F that proves G." If system F is consistent, it cannot output a proof for G; thus, the claim G makes is factually accurate, rendering it true but mechanically unprovable. Furthermore, Gödel demonstrated that system F cannot output a proof of its own consistency. This result fractured David Hilbert's 1900 vision of utilizing a weaker, strictly "finitistic" logical subsystem to definitively prove that the axioms of arithmetic are entirely free of contradictions David Hilbert, "Mathematical Problems," Bulletin of the American Mathematical Society 8 (1902): 437–479.. If the full, powerful system F lacks the mechanical capacity to verify its own consistency, Hilbert's weaker finitistic subsystem is definitively incapable of accomplishing the task. Gödel's work established a hard mechanical boundary, asserting that truth and provability are distinct concepts in classical mathematics.

@@ -220,7 +242,7 @@

- Alan Turing used an abstraction of a computing machine, also described as a clerk working at a desk with pen and squares on paper while following a procedure, to prove that no primary analyzer program can universally decide whether a second analyzed program will halt when it is run Ibid.. An answer to this halting problem (specifically asserting "The analyzed machine halts" or "The analyzed machine does not halt") would indeed be a statement in first-order logic. Thus, by showing no analyzer can universally make such a determination, Turing proved that no decider could exist for the Entscheidungsproblem. + Alan Turing used an abstraction of a computing machine, also described as a clerk working at a desk with pen and squares on paper while following a procedure, to prove that no primary analyzer program can universally decide whether a second analyzed program will halt when it is run Turing, "On Computable Numbers.". An answer to this halting problem (specifically asserting "The analyzed machine halts" or "The analyzed machine does not halt") would indeed be a statement in first-order logic. Thus, by showing no analyzer can universally make such a determination, Turing proved that no decider could exist for the Entscheidungsproblem.

@@ -231,9 +253,9 @@ - M( x ){ - if( H(x ,x) == 'Y' ) while( true ); - else if( H(x ,x) == 'N' ) return; + M(x){ + if( H(x ,x) == 'Y' ) while(true); + else return; } Code . The diagonal machine that defeats a supposed halting decider @@ -244,7 +266,7 @@

- While Gödel, Church, and Turing established the primary boundaries of computation, they did not work in a vacuum. During this period, the broader academic community worked to synthesize the definitive mechanics of effective calculability. Jacques Herbrand and Gödel formalized general recursive functions between 1931 and 1934 Kurt Gödel, "On Undecidable Propositions of Formal Mathematical Systems," mimeographed lecture notes, Institute for Advanced Study, Princeton, 1934.. Emil Post independently defined "Finite Combinatory Processes" in 1936, outlining a theoretical architecture functionally identical to Turing's model Emil L. Post, "Finite Combinatory Processes—Formulation 1," The Journal of Symbolic Logic 1, no. 3 (September 1936): 103–105.. Stephen Kleene subsequently unified these disparate threads, proving the strict mathematical equivalence of Church's lambda calculus and Herbrand-Gödel recursive functions Stephen C. Kleene, "General Recursive Functions of Peano Numbers," Mathematische Annalen 112 (1936): 727–742.. Turing supplied the remaining leg himself, showing his machines equivalent to the lambda calculus Alan M. Turing, "Computability and λ-definability," The Journal of Symbolic Logic 2, no. 4 (December 1937): 153–163.. + While Gödel, Church, and Turing established the primary boundaries of computation, they did not work in a vacuum. During this period, the broader academic community worked to synthesize the definitive mechanics of effective calculability. Jacques Herbrand and Gödel formalized general recursive functions between 1931 and 1934 Kurt Gödel, "On Undecidable Propositions of Formal Mathematical Systems," mimeographed lecture notes, Institute for Advanced Study, Princeton, 1934.. Emil Post independently defined "Finite Combinatory Processes" in 1936, outlining a theoretical architecture functionally identical to Turing's model Emil L. Post, "Finite Combinatory Processes—Formulation 1," The Journal of Symbolic Logic 1, no. 3 (September 1936): 103–105.. Stephen Kleene subsequently unified these disparate threads, proving the strict mathematical equivalence of Church's lambda calculus and Herbrand-Gödel recursive functions Stephen C. Kleene, "General Recursive Functions of Natural Numbers," Mathematische Annalen 112 (1936): 727–742.. Turing supplied the remaining leg himself, showing his machines equivalent to the lambda calculus Alan M. Turing, "Computability and λ-definability," The Journal of Symbolic Logic 2, no. 4 (December 1937): 153–163..

@@ -269,7 +291,7 @@

- For Turing's purposes working on the Entscheidungsproblem, establishing functional equivalence between algorithms and Turing Machine programs was sufficient. However, when the Turing Machine serves as a foundational model for computation theory, we are led to ask another question: whether the Turing Machine is representative of modern architectures, and to the extent it differs, how this would affect the applicability of computation theoretic results. That question occupies the remainder of this chapter, and the answer arrived at is that neither side is quite fit to be compared to the other as it stands. + For Turing's purposes working on the Entscheidungsproblem, establishing functional equivalence between algorithms and Turing Machine programs was sufficient. However, when the Turing Machine serves as a foundational model for computation theory, we are led to ask another question: whether the Turing Machine is representative of modern architectures, and to the extent it differs, how this would affect the applicability of computation theoretic results. That question is addressed over the chapters that follow, and the answer arrived at is that neither side is quite fit to be compared to the other as it stands.

@@ -293,20 +315,19 @@

- Notice where the failure actually sits. Expanding the memory available to a process is handled, and handled well. What is missing is the ability to continue rather than to terminate, and supplying that is software's work, much as Python supplies Integers of arbitrary range over a machine whose Integers are of fixed width. The difference is one of motive. Python's arbitrary range Integers are a convenience offered to the programmer, whereas here the software would be compensating for a bound the architecture imposed. This is the shape of the whole problem in miniature: an architecture is made to look complete by software written to work around it, rather than being complete by virtue of its own instructions. + Notice where the failure actually sits. Expanding the memory available to a process is handled, and handled well. What is missing is the ability to continue rather than to terminate, and supplying that is software's work, much as Python supplies Integers of arbitrary range over a machine whose words are of fixed width. The difference is one of motive. Python's arbitrary range Integers are a convenience offered to the programmer, whereas here the software would be compensating for a bound the architecture imposed. This is the shape of the whole problem in miniature: an architecture is made to look complete by software written to work around it, rather than being complete by virtue of its own instructions.

- The Turing Machine does not escape criticism either, and its defect is the more serious of the two. Its head must read and react at every step, and the only place a controller has to hold what it has seen is a branch in its own control path. So the Turing Machine uses its controller as memory. When we come to analyze a machine that reverses a string, we will find the consequence: the number of states and arcs required grows exponentially against the width of a machine word, and for a word of any practical size the controller cannot be built at all. A machine room operator can mount another reel of tape. Nobody can mount a larger controller, because the controller is finite by definition. The tape's limit was handed outside the machine and dealt with there. The controller's limit is sealed inside the model, where nothing can reach it. + The Turing Machine does not escape criticism either, and its defect is the more serious of the two. Its head must read and react at every step, and the only place a controller has to hold what it has seen is a branch in its own control path. So the Turing Machine uses its controller as memory. When we come to analyze a machine that reverses a string, we will find the consequence: the number of states and arcs required grows exponentially with the width of a machine word, and for a word of any practical size the controller cannot be built at all. A machine room operator can mount another reel of tape. Nobody can mount a larger controller, because the controller is finite by definition. The tape's limit was handed outside the machine and dealt with there. The controller's limit is sealed inside the model, where nothing can reach it.

- To complete the Turing Machine story, then, we will transform the Turing Machine into a modern architecture in a stepwise fashion, while ensuring that at each step the modifications are inconsequential to computation theoretic existence proofs and complexity class results. The transformation does not run in one direction only. On the Turing Machine side, the controller has to stop being used as memory. On the modern side, the fixed widths an architecture stipulates, of an address and of an Integer, are what have to give way. The two meet in the middle, and the machine we arrive at is less strange than that might suggest. It separates the control path from the data path, it holds an instruction table, and it has a small register file. It looks modern. The differences from what we currently build are real but few, and the point of the exercise is that we could build it. + To complete the Turing Machine story, then, we will transform the Turing Machine into a modern architecture in a stepwise fashion, while ensuring that at each step the modifications are inconsequential to computation theoretic existence proofs and complexity class results. The transformation does not run in one direction only. On the Turing Machine side, the controller has to stop being used as memory. On the modern side, the fixed widths an architecture stipulates, those of an address and of an Integer, are what have to give way. The two meet in the middle, and the machine we arrive at is less strange than that might suggest. It separates the control path from the data path; it holds an instruction table; and it has a small register file. It looks modern. The differences from what we currently build are real but few, and the point of the exercise is that we could build it.

- The computer design abstraction stack