From: Thomas Walker Lynch Date: Fri, 21 Aug 2026 18:31:08 +0000 (+0000) Subject: . X-Git-Url: https://git.reasoningtechnology.com/%28%5B%5E?a=commitdiff_plain;h=2779b53261f681ab63f3743228d5c6cad9ec15f1;p=TM-2026 . --- diff --git a/argparse b/argparse new file mode 100644 index 0000000..e69de29 diff --git a/document/book/TM-2026.html b/document/book/TM-2026.html index 0cf578c..80755cf 100644 --- a/document/book/TM-2026.html +++ b/document/book/TM-2026.html @@ -37,7 +37,7 @@ Exordium -

My colleagues in computer arithmetic work tirelessly to confirm Schopenhauer's thesis, so I am left to chose among only unfunded inconsequential problems that no one else wants.

+

My colleagues in computer arithmetic work tirelessly to confirm Schopenhauer’s thesis, so I am left to chose among only unfunded inconsequential problems that no one else wants.

So perhaps we can discuss this one. Turing began with a Naturalist model, a clerk at a desk, yet the machine he described cannot be built. What follows completes the foundation he began. The subject of Computational Naturalism lies at the intersection of number theory, numerical analysis, computation theory, and computer architecture.

@@ -46,12 +46,55 @@ - + + Preface + +

The IEEE Std 754 floating-point standard equips the numerical analyst with the tools to design platform independent algorithms with predictable numerical behavior. Though the principal founder of the standard did commit one sin. He allowed for the expanded precision of the intermediate calculations on the Intel 8087 as a matter of backwards compatibility. He apparently succumbed to that final temptation because he had been part of that project. Consequently, the standard does not provide universal platform independence.

+ +

One of the features of the standard is its support for a limiting arithmetic. If a computation divides by zero, an infinity flag can be substituted as a result. Then if the reciprocal is taken, zero returns. Zero and infinity can be signed, indicating the directions of the overflow and underflow. The substitution flags allow the program doing the computation to continue after what otherwise would be an interrupt exception. However, the intention is to give the numeric analyst tools, not to allow for programs that experience exception conditions to give right answers anyway. In fact, the error, i.e. the distance to the exact result, most commonly increases rapidly after an exception indicating substitution enters the computation.

+ +

Bill Kahan was said principal founder, and the standard happened because he is a staunch, sometimes overbearing, advocate of predictable numerical operations. In my experience in using the standard as a numerical analyst I was struck by how difficult it was to design accurate computations even when given predictable operations. It seems that another layer was needed for helping out the compiler, and I worked on this, and created a manual/automated process for the development of the transcendental functions by my numerics team at AMD. Then I arranged to meet Bill to discuss it.

+ +

We had lunch in Berkeley, and I stressed the point that rather than stopping with operations and limiting arithmetic, that an exception during an operation was merely an excuse to do more computation. Clearly the concept resonated with Bill, as he stopped short after I said it. Also, not long after, Bill gave an interview to Dr. Dobb’sJack Woehr, "A Conversation with William Kahan," Dr. Dobb’s Journal, November 1997, archived at https://jacobfilipp.com/DrDobbs/articles/DDJ/1997/9711/9711a/9711a.htm. A related interview, "An Interview with the Old Man of Floating-Point," February 20, 1998, is sometimes cited alongside it. and there at the top he was quoted repeating my words, "Exceptions are opportunities for extra computation."

+ +

Occasionally a student will come to me to share a great new discovery, that isn’t so great, and isn’t so new. Almost always the idea is not yet mature. Understandably a student’s ideas are sometimes those of a person making observations while not realizing he is on a well trodden path. It is human nature to come to the conclusion that the unknown does not exist. This is not to say that professors do not harvest new ideas from students. My advice to such a student with such a discovery, is to first publish it, perhaps file a provisional patent, or at least write up and send it in an email rather than walking up and telling me about it, and then watch as the concept evolves over time and think what caused that evolution. Of course I was also a student. Hence, the Lynch-Swartlander adder patent was filed in my name (and the names of AMD design team, after all there was a cash reward for those). Also the original publication of High Radix Online Arithmetic at a small conference is in my name.

+ +

Credit and showing proper respect to both students and the hard work that got professors tenure is not as easy of a thing to do as it might seem. I was once asked to present a student’s work at a conference in France, and was so anxious to talk about my own work so as to attract funding, that I nearly left the summary of hers out. I remain embarrassed over this to this day. Though I am sure her colleagues properly appreciate her work. Worse mistakes than this have been made. And what could I possibly have been thinking, a small conference on applied number theory in France is more than a world apart from a Silicon Valley tech meeting.

+ +

Of course I talked to Bill to ask about the quote. I calculated that I had gained a little leverage and thought to use it to perhaps get the support for the project. His backing would help the venture pitch. Bill was obviously quite sincere when he explained that the interviewer had edited the interview to make it dramatic, and that he was himself surprised upon reading it. And of all people to offer an explanation. He could easily have instead said that he had spent most of his career using exceptions as an excuse for more computation, and he would have been right about that. I gained some respect for Bill at that time, though alas, there was no project endorsement. That might be my fault, as come to think of it, I never directly asked for it. Furthermore, nothing in that Dr. Dobbs interview discussed the content, rather it was merely the sound bite enticing the next question: what is that further computation? This book discusses this topic.

+ +

John Gustafson is another person who has put a lot of thought into how to formulate a valuable floating-point standard, and he now has a competing proposal. John also brought up the topic of the nature of error shortly before publishing The End of Error. He has improved and evolved that to the Posit, which abandons the variable width operands of his earlier unums in favor of a fixed width whose internal field boundaries move. This reminds me of the denorms of Std 754, but it is more graceful. Precision tapers away smoothly to the smallest representable value. John was polite and asked me about the method I had used to achieve accuracy in microcode. I answered describing High Radix Online Arithmetic and with the importance of variable precision, but I could not formulate the rest of the thoughts. I had learned a great deal while doing formal analysis of the algorithms I had put on the Am29050, Am486, and K5 processors at AMD, yet when I tried to articulate this, I could not explain it. It is articulated better in this book.

+ +

Many others have given thought to the problem of errors in computation and have made effective proposals. Moore originally introduced interval methods, where two numbers bound a true value. Kulisch and Miranker attempted to formalize computer arithmetic, and started by reasoning about a screen that lay over the real number space, which is a route to the interval methods they embedded in their XSC languages. Boehm and Cartwright introduced a concept of Constructive Real Numbers, where precision of computation was extended upon demand. And through Stephen Wolfram’s vision, Mathematica provides a complete symbolic math manipulation package, where a symbolic expression can be simplified. Upon a call for numeric resolution Mathematica then tracks loss of accurate precision through significance arithmetic and repeats the computation at increased precision if the accuracy of the result is not sufficient. In my own work at AMD, my approach was to perform an analysis step first, where required precision was derived, then followed by variable precision computation.

+ +

However, all of these methods are missing a key insight. That the further computation that the exception invites us to engage in, is part of a richer second order system. The various systems mentioned above remain in the first order, though perhaps with a hint of entering a second order analysis.

+ +

In practical computing the error algebra of IEEE Std 754 can quickly mount due to the substitution flags, and the system tends towards the catch-all indeterminate value, the so-called NaN. This effect leaves some computer scientists wishing to extend the system. However this is not a failure of the system as preventing this was not the design intent. Extensions to the system do not usually fare much better, because error tag algebra can not replace computation.

+ +

With interval methods, the intervals will encompass error causing singularities that otherwise would not have manifested. However, the intervals also tend to pessimistically widen and then pull in extraneous singularities. Also, as for all these methods, exceptions can happen at run time without the code designer being aware of their nature or perhaps of the potential for it to happen. And when an exception does occur, it tends to be handled ad hoc in the manner as in conventional computation, for example by exiting the program as the interval system does not provide further guidance.

+ +

The Constructive Real is reminiscent of a limit, it will lazy evaluate for increased precision until obtaining a sufficiently accurate result. A limit concept is indeed key when moving from first order computation to a second order, but such a second order system is not built in. As for interval methods, an error could surprise an algorithm designer when it appears at run time, and thus the code might not be prepared to handle it. However, there is an interesting aspect here, the expression syntax trees are available to the exception handler, or at least could be, so a layer could be built on top.

+ +

Mathematica uses symbolic manipulation of expressions combined with significance arithmetic, so numeric resolution can fail, and then the user is left holding the error as a result. However, similarly to how the Constructive Real system has syntax trees, the exception handler can have access to the unresolved symbolic expression, which the user could perform more computation with if he knew what to do with it. This is interesting.

+ +

There is something more fundamental going on here that all of these approaches are dancing around. There are limits to what is possible with computing and even logic itself. Attaching number representations to the structures does not change this fact. Hence, this book will start not with numerical analysis, but with the foundation of mathematics and computation theory.

+ +

There is a first edition of this book.Thomas Walker Lynch, Tom’s Turing Complete Architecture: Volume 1, Foundation and TM Library Interface (Independently published, April 24, 2023), 230 pp., ISBN 979-8392389087. The Lisp library described in that edition was released on Quicklisp. The same concepts are described in those pages, but the story is not as well told, and no one has come full circle to say that it made sense. One colleague who reviewed it admitted he was bewildered by it, and asked whether I was proposing that we build Turing Machines. After all, we are the music makers,Arthur O’Shaughnessy, "Ode", in Music and Moonlight (London: Chatto and Windus, 1874). so to speak; and we write notes in transistors and layout. So then, my colleague Kelvin rightly asked, "What is the proposal here?" I have yet to give him an answer. Perhaps this second edition will help.

+ +

So as to address Kelvin’s question directly, and to prepare other readers who are familiar with my computer architecture work, let me say that there is no hardware specification for a new proposed commercial machine in this book. This book is not proposing to build a Turing Machine in hardware. This is an absolute. Building a Realizable Machine might be interesting. It is true that the organization for it is described in this book. Yet this is certainly not the proposal here. Rather this book explains and justifies a set of features that are planned to be incorporated into a commercial machine, and yes, that is the machine I was pitching for funding to develop in 2005.

+ +

Though a practical commercial processor design is not provided in these pages, there is a saving grace, i.e. a practical contribution. There is a formally grounded iterator library described and available for download from GitHub. This library appeared in the first edition implemented in Lisp (SBCL, Common Lisp, rather than Scheme) and released on Quicklisp. That library is described in an appendix. Lisp is in a sense pure and formal statements can be made with it, so writing the library in Lisp made sense. Certain improvements have been made since the first edition, and this volume introduces and discusses the Python version.

+ + + + + + Conventio -

All communication starts from the knowledge held in common between the author and reader. This chapter is a first meeting and a search for that commonality. It is not the start of the thesis of the book, that being Computational Naturalism. That topic doesn't even come up. A reader who skipped these pages would be more likely to misread what follows, but would not miss any of the argument. What follows is the notation this book is written in and the words it takes from mathematics and logic. There are some terms introduced by this book and mentioned here, and those are so marked.

+

All communication starts from the knowledge held in common between the author and reader. This chapter is a first meeting and a search for that commonality. It is not the start of the thesis of the book, that being Computational Naturalism. That topic doesn’t even come up. A reader who skipped these pages would be more likely to misread what follows, but would not miss any of the argument. What follows is the notation this book is written in and the words it takes from mathematics and logic. There are some terms introduced by this book and mentioned here, and those are so marked.

- + Notation

In math spans and blocks, each container, such as a sequence or a set, is represented with a single capital letter, even when in Greek. Each object that is not a container is represented with a single lower case letter. Character pairs or full symbol names can also be used to represent math objects, in which case the same convention is followed for the first letter of the symbol, while the remaining letters are lower case. The context will make it clear if a non-letter Unicode character represents a container or a non-container.

@@ -88,44 +131,44 @@ Code . A hyphen binding tighter than an underscore within an identifier -

Throughout, this text uses native Unicode characters rather than escape sequences. Where a mathematical or Greek symbol is wanted, the character itself appears, so the text carries 'δ' rather than \delta. This might lead to some small typesetting variations compared to what MathJax might otherwise have output.

+

Throughout, this text uses native Unicode characters rather than escape sequences. Where a mathematical or Greek symbol is wanted, the character itself appears, so the text carries ‘δ’ rather than \delta. This might lead to some small typesetting variations compared to what MathJax might otherwise have output.

-

The Unicode middle dot, ·, is used as an ad hoc namespace operator in identifiers. Hence N·x would be the variable x from the N namespace. This is a typographic symbol with a long history of use as a separator. It appears in dictionaries where words are broken into syllables, e.g. 'op·er·a·tor'. It is a letter of the Catalan alphabet, the punt volat, which distinguishes the geminate l·l of 'col·legi' from the digraph ll. And Japanese uses it, as the nakaguro, to separate the components of a transliterated foreign name. It is accepted in identifiers by modern C compilers,C11 permits the character anywhere in an identifier, including as the first character: it falls in the 00B7−00BA range of Annex D.1, and appears in none of the ranges disallowed initially in D.2. C23 restates the rule in terms of Unicode UAX #31, where the character belongs to Other_ID_Continue and so remains permitted, though no longer in first position. ISO/IEC 9899:2011, Annex D; ISO/IEC 9899:2024, §6.4.2. and it is part of the RT·gcc compiler mods as an ad hoc namespace operator.

+

The Unicode middle dot, ·, is used as an ad hoc namespace operator in identifiers. Hence N·x would be the variable x from the N namespace. This is a typographic symbol with a long history of use as a separator. It appears in dictionaries where words are broken into syllables, e.g. ‘op·er·a·tor’. It is a letter of the Catalan alphabet, the punt volat, which distinguishes the geminate l·l of ‘col·legi’ from the digraph ll. And Japanese uses it, as the nakaguro, to separate the components of a transliterated foreign name. It is accepted in identifiers by modern C compilers,C11 permits the character anywhere in an identifier, including as the first character: it falls in the 00B7−00BA range of Annex D.1, and appears in none of the ranges disallowed initially in D.2. C23 restates the rule in terms of Unicode UAX #31, where the character belongs to Other_ID_Continue and so remains permitted, though no longer in first position. ISO/IEC 9899:2011, Annex D; ISO/IEC 9899:2024, §6.4.2. and it is part of the RT·gcc compiler mods as an ad hoc namespace operator.

 - + Usage -

In the RT coding culture, we say that argument values are given to a function. Functions do not take arguments, as it is the caller that has authority over making the call, and thus over sending the arguments. A function is only said to accept or reject a value in the context of guard code evaluating argument validity, something it has authority to do. Similarly, machines, factories, and software make objects by assembling components they are given; they do not create them, since bringing components into being is not within an engineer's authority. That leaves 'creation' to imply a more fundamental synthesis outside the scope of engineering, perhaps done by artists or supernatural beings.

+

In the RT coding culture, we say that argument values are given to a function. Functions do not take arguments, as it is the caller that has authority over making the call, and thus over sending the arguments. A function is only said to accept or reject a value in the context of guard code evaluating argument validity, something it has authority to do. Similarly, machines, factories, and software make objects by assembling components they are given; they do not create them, since bringing components into being is not within an engineer’s authority. That leaves ‘creation’ to imply a more fundamental synthesis outside the scope of engineering, perhaps done by artists or supernatural beings.

-

I have experimented with modern gender forms in technical language in past writings. For example, I used the plural-as-singular style in one piece, and was applauded by some, though categorized as illiterate by others. Since then, I have evolved a writing style that emphasizes using roles as subjects: the mathematician, the author, the programmer. Such subjects are singular, so for grammatical agreement, I use the inclusive he. It is structurally much cleaner to use he as inclusive of all people than it is to gray out an entire category of plural agreement pronouns by forcing words such as 'they' to take on a singular form. I use 'a person' when the subject is an unknown third party, who might even be an AI. I prefer this over the 'one' of 'One does this, or one does that', so that 'one' can be reserved to unambiguously refer to the number.

+

I have experimented with modern gender forms in technical language in past writings. For example, I used the plural-as-singular style in one piece, and was applauded by some, though categorized as illiterate by others. Since then, I have evolved a writing style that emphasizes using roles as subjects: the mathematician, the author, the programmer. Such subjects are singular, so for grammatical agreement, I use the inclusive he. It is structurally much cleaner to use he as inclusive of all people than it is to gray out an entire category of plural agreement pronouns by forcing words such as ‘they’ to take on a singular form. I use ‘a person’ when the subject is an unknown third party, who might even be an AI. I prefer this over the ‘one’ of ‘One does this, or one does that’, so that ‘one’ can be reserved to unambiguously refer to the number.

-

In the prior edition of this book, the preface included a discussion on the meaning of the word 'may' according to RFC 2119, which fixes the requirement level key words used in specification writing. There is an important distinction between the 'may' of options or permission, and the more colloquial 'may' of probability. However, distinguishing between the two was too much of an ask of readers, most of whom skip the preface anyway. So in this edition, I avoid the temptation to use 'may' and replace it with a direct statement of what I mean. "It is of high probability that..." or "There are options for...". Directly saying what is meant, who would have thought of it? The RFC series has since narrowed the word further. RFC 8174 amended RFC 2119 so that the key words carry their defined meanings only when written in capitals, which leaves a lower case 'may' in an RFC with no normative force at all.Scott Bradner, Key words for use in RFCs to Indicate Requirement Levels, RFC 2119, March 1997. Barry Leiba, Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words, RFC 8174, May 2017.

+

In the prior edition of this book, the preface included a discussion on the meaning of the word ‘may’ according to RFC 2119, which fixes the requirement level key words used in specification writing. There is an important distinction between the ‘may’ of options or permission, and the more colloquial ‘may’ of probability. However, distinguishing between the two was too much of an ask of readers, most of whom skip the preface anyway. So in this edition, I avoid the temptation to use ‘may’ and replace it with a direct statement of what I mean. "It is of high probability that..." or "There are options for...". Directly saying what is meant, who would have thought of it? The RFC series has since narrowed the word further. RFC 8174 amended RFC 2119 so that the key words carry their defined meanings only when written in capitals, which leaves a lower case ‘may’ in an RFC with no normative force at all.Scott Bradner, Key words for use in RFCs to Indicate Requirement Levels, RFC 2119, March 1997. Barry Leiba, Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words, RFC 8174, May 2017.

Dates and times are given in the ISO 8601 format. The components run from the most-significant to the least-significant, year, then month, then day, in the same order the digits of a Hindu-Arabic numeral run. Two consequences follow from that ordering, and both are the reason for choosing it. A lexicographic comparison of two such dates agrees with their chronological comparison, so a list of them sorts correctly with no collation rules particular to dates. And the format is unambiguous, whereas a date such as 03/04/2026 is read as the third of April by some readers and the fourth of March by others. A time of day, when one is given, follows the date, and a trailing Z marks the time as UTC. Hence 2026-06-01 08:28:00Z. ISO 8601 itself separates the date from the time with a T, which is difficult to read. RFC 3339 anticipates the objection and permits a space in its place, and that is the option taken here.Graham Klyne and Chris Newman, Date and Time on the Internet: Timestamps, RFC 3339, July 2002, §5.6. Where a date appears without a time, the date alone is intended, e.g. 2026-06-01. As per the standard, if the day is not to be specified, it is simply dropped, e.g. 2026-06.

 - + Number types

Several words in this book carry a formal definition that displaces their ordinary English sense, and those are capitalized: Peano Number, Integer, Rational, Real. The word real is the best example of why this is done. It was too much of an ask by mathematics to take a common English adjective and hold it to a technical meaning, and a reader is entitled to know which sense is in play. So a Real is the object mathematics defines, and a real problem or a real difficulty is the ordinary English word. Integer suffers no such ambiguity. It is capitalized for consistency with its siblings.

The leading capital letter on the mathematical number types does not distinguish the classical notion from the one constructed herein, because this book denies that the two are distinct in the first place. A Real is a Real whether a mathematician defines it or a machine produces the definition.

-

What mathematics calls the natural numbers are called Peano Numbers in this book. A Peano Number is what a Peano Machine outputs. This convention frees the word 'Natural' to refer to the philosophy only, so a reader never has to work out which of the two is meant. Note that herein Peano Numbers are taken to begin at zero, while Counting Numbers begin at one, and the two are not the same.Peano's own axioms began at one. Modern presentations of Peano arithmetic begin at zero, and that is the convention followed here. Giuseppe Peano, Arithmetices principia, nova methodo exposita (Turin: Bocca, 1889), §1. An Integer, a Rational, and a Real are likewise the outputs of machines, each constructed in its turn.

+

What mathematics calls the natural numbers are called Peano Numbers in this book. A Peano Number is what a Peano Machine outputs. This convention frees the word ‘Natural’ to refer to the philosophy only, so a reader never has to work out which of the two is meant. Note that herein Peano Numbers are taken to begin at zero, while Counting Numbers begin at one, and the two are not the same.Peano’s own axioms began at one. Modern presentations of Peano arithmetic begin at zero, and that is the convention followed here. Giuseppe Peano, Arithmetices principia, nova methodo exposita (Turin: Bocca, 1889), §1. An Integer, a Rational, and a Real are likewise the outputs of machines, each constructed in its turn.

 - + Fields of study -

Number Theory consists of the analysis of Peano Number related structures. It gives properties to these structures, e.g. the familiar 'odd' and 'even'. It gives properties to the relationship between numbers and operations on those numbers, e.g. proving that an odd number squared is odd. It deals with the solution of mixed integer equations, and systems of such equations, and sets the rules for modulus arithmetic. For example, encryption algorithms belong to Number Theory.

+

Number Theory consists of the analysis of Peano Number related structures. It gives properties to these structures, e.g. the familiar ‘odd’ and ‘even’. It gives properties to the relationship between numbers and operations on those numbers, e.g. proving that an odd number squared is odd. It deals with the solution of mixed integer equations, and systems of such equations, and sets the rules for modulus arithmetic. For example, encryption algorithms belong to Number Theory.

Numerical Analysis is built on two axioms. The first axiom limits computing to vectors of Peano Numbers, where a vector represents a number. A common representation is a number pair, where one number of the pair scales the other number, thus making it possible to represent fractional values. The second axiom states the actual intention of the programmer is to perform computation over a Real field. As Real Numbers fall on a continuum, it follows that some Real Numbers can not be represented with a vector of Peano Numbers. The objective of Numerical Analysis is then to establish the relationship between Peano Number results with the corresponding exact Real Number results. The difference between the two is called error. People who work in numerical algorithm design often spend a great deal of time trying to keep some metric of error low over a given domain.

Computation Theory consists of proofs over what can, and cannot be computed, the time and space complexity of algorithms, and the fundamental structure of computation. Number Theory is a kind of Computation Theory applied to Peano Structures before Church, Turing, and their contemporaries gave computation theory its name.

 - + Symbol

A math object is anything mathematics is willing to name: a number, a set, a function, a formula, or a symbol. Nothing further is meant by it. A value, where the term is used without qualification, is synonymous with math object.

@@ -136,7 +179,7 @@

A symbol scope, or when the context of the usage is clear, simply scope, is a set of evaluation contexts. The members of the set are specified by the author who presents the math. A commonly specified scope is that of the math context. This is what most people will think of when the terms we are about to discuss are used. Applying this reading will produce correct definitions, though not the only possible definitions.

-

A symbol is a designated math object, where each symbol is distinct from every other symbol within scope, and such that each symbol can be represented. We don't write down a symbol directly. What a person sees on a piece of paper, or on the computer screen, is an instance of a symbol representation. Most mathematicians prefer to represent each symbol using a single letter of the English, Greek, or sometimes some other alphabet. Mathematicians don't often use the terminology of representation and instance, but might instead refer to an instance of a symbol representation as a sign, a letter, or a mark.Philosophy calls this the distinction between a type and its tokens, following Peirce. The fit is close but not exact, as that pair has no middle layer answering to what is here called a representation. Charles S. Peirce, Collected Papers, vol. 4, §537.

+

A symbol is a designated math object, where each symbol is distinct from every other symbol within scope, and such that each symbol can be represented. We don’t write down a symbol directly. What a person sees on a piece of paper, or on the computer screen, is an instance of a symbol representation. Most mathematicians prefer to represent each symbol using a single letter of the English, Greek, or sometimes some other alphabet. Mathematicians don’t often use the terminology of representation and instance, but might instead refer to an instance of a symbol representation as a sign, a letter, or a mark.Philosophy calls this the distinction between a type and its tokens, following Peirce. The fit is close but not exact, as that pair has no middle layer answering to what is here called a representation. Charles S. Peirce, Collected Papers, vol. 4, §537.

A symbol can be bound to any value, including another symbol. This binding can be expressed as a predicate, \text{bind}(s, v), where s is the symbol and v is the value it is bound to. Each distinct pair of arguments gives a distinct binding, so a binding is an object in its own right, and two bindings of the same symbol to different values are two objects rather than one object altered.

@@ -152,17 +195,17 @@ - + Logic - + Propositional calculus

A proposition is a statement that is given no arguments and carries a truth value, that being one of exactly two things, written true and false. A proposition is not predicated upon anything, so settling it requires nothing to be looked up and no value to be supplied. "The moon is made of green cheese" is a proposition, and it is false. "Seven is prime" is a proposition, and it is true. In either case the statement stands complete as it is written. Nothing in the calculus that follows looks inside a proposition or has any means of altering one. A proposition enters the calculus carrying its truth value, and the calculus makes use of nothing else about it.

The connectives are symbols, and instances of their representations are these: ∧ (and), ∨ (or), ¬ (not), → (implies), and ↔ (if and only if). Alongside them stand the parentheses, ( and ), which group. Each connective is fixed, in that what it does is settled once for the whole calculus and nothing later on is at liberty to change it. What each one does is combine truth values to give a truth value, so ∧ gives true exactly when both of the truth values given to it are true, ¬ gives the truth value it was not given, and so on for the rest.

-

Beside the connectives stand the proposition symbols, whose representations are commonly p, q, and r. These are variables. A proposition symbol has no meaning of its own, and marks a place where a proposition is later bound to it. It is the only kind of symbol in this calculus that is not fixed.The letters follow the notation convention of section , applied to whatever the symbol marks a place for. A proposition is not a container, so the letter is lower case. A formula is a string and therefore a container, so the letters standing for formulas below are capitals.

+

Beside the connectives stand the proposition symbols, whose representations are commonly p, q, and r. These are variables. A proposition symbol has no meaning of its own, and marks a place where a proposition is later bound to it. It is the only kind of symbol in this calculus that is not fixed.The letters follow the notation convention of section , applied to whatever the symbol marks a place for. A proposition is not a container, so the letter is lower case. A formula is a string and therefore a container, so the letters standing for formulas below are capitals.

Symbol instances are strung together according to the formation rules, and what the rules build is called a formula. The rules are three, and the third is what closes the definition. The Greek letters Φ and Ψ stand for whole formulas below, and ∘ stands for any one of the binary connectives. All three belong to this discussion rather than to the calculus being discussed.

@@ -181,7 +224,7 @@

A computer scientist would see the formation rules as forming a grammar, and a parser built from them would sort strings into two categories, the well formed formula and the malformed formula. Take the string p ¬ ∧ q, which would not parse. However, a logician would point out that this string is not a result of applying the formation rules, so it is simply not a formula at all, and there is no second category called malformed formula for it to fall into. Yet the traditional term used by logicians for a formula is well formed formula, shortened to wff, and along with formula all three are used interchangeably. Herbert B. Enderton, A Mathematical Introduction to Logic, 2nd ed. (San Diego: Harcourt/Academic Press, 2001), §1.1, which introduces the three words as alternatives for one thing. Texts that keep the adjective and texts that drop it are describing the same objects. Meanwhile the philosophers have spent two centuries telling us that naming a thing this way implies the existence of its opposite. - That a determination carries its own negation is Hegel's point, taken from Spinoza's omnis determinatio est negatio. G. W. F. Hegel, Wissenschaft der Logik (Nuremberg: Schrag, 1812–1816), Book One, on determinate being; Baruch Spinoza to Jarig Jelles, June 2, 1674, Letter 50. A C++ committee borrowed the adjective for a well formed program and then found it necessary to name the opposite, which became the ill formed program. ISO/IEC 14882, [defns.well.formed] and [defns.ill.formed]. There the pair does work, because a compiler is required to say which side a given input falls on. + That a determination carries its own negation is Hegel’s point, taken from Spinoza’s omnis determinatio est negatio. G. W. F. Hegel, Wissenschaft der Logik (Nuremberg: Schrag, 1812–1816), Book One, on determinate being; Baruch Spinoza to Jarig Jelles, June 2, 1674, Letter 50. A C++ committee borrowed the adjective for a well formed program and then found it necessary to name the opposite, which became the ill formed program. ISO/IEC 14882, [defns.well.formed] and [defns.ill.formed]. There the pair does work, because a compiler is required to say which side a given input falls on. Who do we believe, the logicians or the philosophers? This book accepts the term malformed formula for something intended or presented as a formula, although when the formation rules are applied as a grammar, turns out to have one or more syntax errors.

An interpretation binds a proposition to each proposition symbol of a formula. Since each connective combines truth values and inspects nothing else, an interpretation carries no more information than the truth value each bound proposition holds, and the formula then holds or fails according to those truth values alone. Let p be bound to "the clock line is high" and q to "the enable line is high"; whether (p ∧ ¬q) holds follows from those two truth values and from nothing else. An interpretation does not alter the formula, which was settled before any interpretation was considered, and it does not reach inside a proposition, which was settled before it was bound.

@@ -215,12 +258,12 @@ - + First-order predicate calculus

Now suppose that a proposition could be predicated on something else. A predicate is a logic function. It is given values for its arguments, and it then evaluates to true or to false. Give a predicate its arguments and what results is a proposition, so the predicate is the general case and the proposition is the settled one. \text{prime}(x) is a predicate of one argument, and \text{prime}(7) is true while \text{prime}(8) is false. <(x, y) is a predicate of two arguments, which is how a relation is written. Thus <(3, 5) is true, while <(5, 3) is false. Predicate arguments are drawn from a domain, a set holding at least one member, and that set is whatever mathematics cares to name: the Peano Numbers, the points of a plane, the strings over an alphabet. Nothing requires the arguments to be truth values, and nothing forbids it. What is required is that the result be a truth value.

-

A property is a predicate of one argument. It is not a new kind of object; it is the one-argument case of the predicate already defined. \text{even} is a property of the Peano Numbers, holding of 4 and failing of 5, and \text{prime} is another. To give a property over a domain is to mark off which members of that domain it holds of, so the properties of a domain stand in exact correspondence with the sub-collections of it, and a domain of n members carries 2^n properties.This correspondence counts two predicates holding of the same members as the same property, a convention called extensionality. It sets aside the sense in which 'is an equilateral triangle' and 'is an equiangular triangle' are different properties that happen to hold of the same figures. The properties of a domain form a collection of their own, distinct from the domain, so a quantifier ranging over the one is not ranging over the other.

+

A property is a predicate of one argument. It is not a new kind of object; it is the one-argument case of the predicate already defined. \text{even} is a property of the Peano Numbers, holding of 4 and failing of 5, and \text{prime} is another. To give a property over a domain is to mark off which members of that domain it holds of, so the properties of a domain stand in exact correspondence with the sub-collections of it, and a domain of n members carries 2^n properties.This correspondence counts two predicates holding of the same members as the same property, a convention called extensionality. It sets aside the sense in which ‘is an equilateral triangle’ and ‘is an equiangular triangle’ are different properties that happen to hold of the same figures. The properties of a domain form a collection of their own, distinct from the domain, so a quantifier ranging over the one is not ranging over the other.

The first-order predicate calculus also begins with symbols, and its base level holds three groups rather than two. The fixed group gains the quantifiers, ∀ (for all) and ∃ (there exists), alongside the connectives and parentheses already in it. The second group holds the variables an interpretation binds, and there are three kinds of them. A predicate symbol is a variable to be bound to a predicate, and it carries a fixed number of argument places, so that the predicate bound to it is one of that many arguments; its representations are commonly F, G, and R. A name is a variable to be bound to one member of the domain; its representations are commonly a, b, and c. A function symbol is a variable to be bound to a function carrying some fixed number of domain members to one domain member, and it carries that many argument places; its representations are commonly f, g, and s.The capitals are the traditional letters for predicate symbols, and are kept so that a reader can find the literature. The third group is new, and holds the variables a quantifier is able to claim. This calculus has one kind, the domain variable, whose representations are commonly x, y, and z. The proposition symbol is gone, having no work left to do: a predicate symbol given its arguments occupies the place where a proposition once stood.

@@ -239,11 +282,11 @@

So ∀x \, F(x) and ∀x ∃y \, R(x, y) are formulas of the calculus.

-

A quantifier claims the domain variable written after it, and its claim reaches over the formula that rule 3 was applied to and no further. That formula is the quantifier's scope. Where a variable appears within the scope of a quantifier claiming it, that appearance is bound; where it appears within no such scope, that appearance is free.The words bound and free are the standard ones, and are given here so that a reader can find the literature. The word claims is used alongside them in this book because it names the agency. An interpretation binds nothing to a bound appearance, because the quantifier is what runs it over the domain, passing the formula over every member in the universal case and over at least one member in the existential case. A free appearance is left for an interpretation to bind. There are these two fates and no third, and no appearance meets both.

+

A quantifier claims the domain variable written after it, and its claim reaches over the formula that rule 3 was applied to and no further. That formula is the quantifier’s scope. Where a variable appears within the scope of a quantifier claiming it, that appearance is bound; where it appears within no such scope, that appearance is free.The words bound and free are the standard ones, and are given here so that a reader can find the literature. The word claims is used alongside them in this book because it names the agency. An interpretation binds nothing to a bound appearance, because the quantifier is what runs it over the domain, passing the formula over every member in the universal case and over at least one member in the existential case. A free appearance is left for an interpretation to bind. There are these two fates and no third, and no appearance meets both.

The fates belong to appearances and not to variables, as two ordinary formulas show. In ∀x \, (F(x) ∧ ∃x \, G(x)) the inner quantifier claims the appearance inside G, and the outer one does not reach past it, so a claim is able to be shadowed by a nearer claim on the same variable. In (F(x) ∧ ∀x \, G(x)) the same variable appears free in the left conjunct and bound in the right. Both are formulas, and both are legible only by tracking appearances one at a time. A calculus that means to avoid the bookkeeping imposes a hygiene condition, requiring that no variable be claimed twice and that no variable appear both free and bound, and it then renames variables as needed to meet the condition. Renaming a bound variable throughout its scope changes nothing, since the quantifier runs it over the domain either way. Renaming a free variable changes what the formula is about.

-

A formula in which no variable appears free, also called a sentence, makes an outright claim once its predicate symbols, names, and function symbols are bound. A formula with a free appearance holds or fails according to what is bound to that appearance, so its truth value varies where the other's does not. So ∀x ∃y \, R(x, y) is one and ∃y \, R(x, y) is not. The distinction has no work to do in the propositional calculus, which holds no quantifier and so no free appearances.

+

A formula in which no variable appears free, also called a sentence, makes an outright claim once its predicate symbols, names, and function symbols are bound. A formula with a free appearance holds or fails according to what is bound to that appearance, so its truth value varies where the other’s does not. So ∀x ∃y \, R(x, y) is one and ∃y \, R(x, y) is not. The distinction has no work to do in the propositional calculus, which holds no quantifier and so no free appearances.

An interpretation of a first-order formula supplies a domain, which holds at least one member.Standard first-order logic does not admit the empty domain, and the reason is visible in the formulas. Over an empty domain ∀x \, F(x) would hold and ∃x \, F(x) would fail, so (∀x \, F(x) → ∃x \, F(x)) would not be valid; it is counted a validity of this calculus, and admitting the empty domain would cost that. It then binds, to each variable of the formula, a value of the kind that variable marks a place for: a predicate over the domain to each predicate symbol, of the matching number of arguments; a member of the domain to each name; a function over the domain to each function symbol; and a member of the domain to each free appearance of a domain variable, passing over the bound appearances, which the quantifier has already claimed.The notation is not settled across the sources surveyed in this book. Hilbert and Ackermann write (x) for the universal quantifier and (Ex) for the existential, and they mark negation with an overbar rather than with ¬. A reader going to their book should be prepared for this. Take the Peano Numbers as the domain and bind < to R: the formula ∀x ∃y \, R(x, y) then states that every Peano Number has a larger one, and under that interpretation it holds. Every appearance in it is claimed by a quantifier, so the only things the interpretation had to give were the domain and the predicate.

@@ -274,7 +317,7 @@ List . The first order predicate logic stack - + Second-order predicate calculus

In the first-order calculus a quantifier claims a domain variable and nothing else. Each predicate symbol is bound by an interpretation, so a person supplies a particular predicate for F and asks what follows, and no formula of that calculus ranges over the predicates themselves. The second-order predicate calculus lifts that restriction by admitting a second kind of claimable variable, the predicate variable, which carries a fixed number of argument places. The two fates are as before, and now apply to this second kind of symbol as well. In ∀F \, Φ the quantifier has claimed F, so no interpretation binds a predicate to it, and the quantifier runs it over the properties of the domain in the same way that ∀x runs a domain variable over the domain itself.

@@ -316,7 +359,7 @@ - + More logic terminology

A formula that holds under every interpretation is valid, and in the propositional calculus such a formula is also called a tautology. A formula that holds under at least one interpretation is satisfiable. So ((p ∧ p) ↔ p) is valid, (p ∧ q) is satisfiable and not valid, and (p ∧ ¬p) is neither. The two are duals: a formula is valid exactly when its negation is unsatisfiable.

@@ -339,116 +382,207 @@

A collection of axioms and rules is sound when everything it derives is valid, so ⊢ never outruns ⊨, and complete when everything valid is derivable, so ⊨ never outruns ⊢.The turnstile is reused later in this book, subscripted with a machine, for the relation between one instantaneous description of a Turing Machine and the next. The subscript keeps the two apart on the page. A calculus together with a collection of axioms is a theory. Completeness is a property of the axioms and rules relative to the validities of a calculus, and is not the same question as whether a theory settles every statement it can express.

-

Finally, the word order counts something different in each of the two places this book uses it, and the two scales are independent of each other. In the predicate calculus, the order counts what a quantifier runs over: a domain member at the first order, and a property of domain members at the second. In the orders of analysis set out in section , the order counts the depth of machines examining machines: running a machine is the first order, examining a machine's definition is the second, and examining an examiner is the third. A second-order analysis of a machine that computes over the first-order predicate calculus is an ordinary thing, and nothing about it is second-order logic. Where a later chapter of this book speaks of second-order arithmetic, it is the orders of analysis that are meant. The same warning applies to metamathematics: a statement about a first-order system is made in the metalanguage, and that alone does not make it a second-order statement.

+

Finally, the word order counts something different in each of the two places this book uses it, and the two scales are independent of each other. In the predicate calculus, the order counts what a quantifier runs over: a domain member at the first order, and a property of domain members at the second. In the orders of analysis set out in section , the order counts the depth of machines examining machines: running a machine is the first order, examining a machine’s definition is the second, and examining an examiner is the third. A second-order analysis of a machine that computes over the first-order predicate calculus is an ordinary thing, and nothing about it is second-order logic. Where a later chapter of this book speaks of second-order arithmetic, it is the orders of analysis that are meant. The same warning applies to metamathematics: a statement about a first-order system is made in the metalanguage, and that alone does not make it a second-order statement.

 - - - - Preface + + Naturalism -

The IEEE Std 754 floating-point standard equips the numerical analyst with the tools to design platform independent algorithms with predictable numerical behavior. Though the principal founder of the standard did commit one sin. He allowed for the expanded precision of the intermediate calculations on the Intel 8087 as a matter of backwards compatibility. He apparently succumbed to that final temptation because he had been part of that project. Consequently, the standard does not provide universal platform independence.

+ + Platonism -

One of the features of the standard is its support for a limiting arithmetic. If a computation divides by zero, an infinity flag can be substituted as a result. Then if the reciprocal is taken, zero returns. Zero and infinity can be signed, indicating the directions of the overflow and underflow. The substitution flags allow the program doing the computation to continue after what otherwise would be an interrupt exception. However, the intention is to give the numeric analyst tools, not to allow for programs that experience exception conditions to give right answers anyway. In fact, the error, i.e. the distance to the exact result, most commonly increases rapidly after an exception indicating substitution enters the computation.

+

Plato (427−347 BC) spoke of an entity with an independent existence he called a Form. A Form is immutable, non-spatial, and non-temporal, and exists in a realm that has no location, called the Realm of Forms. Examples of Forms include mathematical properties (Equality, Circularity, Squareness, Unity, Doubleness), moral concepts (Justice, Courage, Piety, Temperance), aesthetic ideals (Beauty), physical archetypes (Tableness, Bedness, Treeness), and above them all the Good, which Plato held to be what makes the other Forms knowable. Plato then explains that knowledge is the conceptualization of Forms. +

-

Bill Kahan was said principal founder, and the standard happened because he is a staunch, sometimes overbearing, advocate of predictable numerical operations. In my experience in using the standard as a numerical analyst I was struck by how difficult it was to design accurate computations even when given predictable operations. It seems that another layer was needed for helping out the compiler, and I worked on this, and created a manual/automated process for the development of the transcendental functions by my numerics team at AMD. Then I arranged to meet Bill to discuss it.

+

+ As an illustration of the independent existence of Forms, consider what Plato says about geometers in the Republic. They draw a square and a diagonal, and then they argue about it. But they are not arguing about the figure they drew, and everyone participating in the discussion knows this to be the case. They are not asking whether that line, the one in the sand, is straight, nor whether that corner is square, for it plainly is not, and no conclusion they reach depends upon it. The drawn figure is used as an image of the Square itself, and what they say holds of the Square itself whether or not the drawing was made well, or made at all.Plato, Republic VI 510c−511a. The passage places mathematics on the third segment of the divided line, above opinion and below the account that would justify its hypotheses. Aristotle grants the same observation about mathematical practice and gives it the opposite explanation, at Metaphysics XIII 1078a17−31. +

-

We had lunch in Berkeley, and I stressed the point that rather than stopping with operations and limiting arithmetic, that an exception during an operation was merely an excuse to do more computation. Clearly the concept resonated with Bill, as he stopped short after I said it. Also, not long after, Bill gave an interview to Dr. Dobb'sJack Woehr, "A Conversation with William Kahan," Dr. Dobb's Journal, November 1997, archived at https://jacobfilipp.com/DrDobbs/articles/DDJ/1997/9711/9711a/9711a.htm. A related interview, "An Interview with the Old Man of Floating-Point," February 20, 1998, is sometimes cited alongside it. and there at the top he was quoted repeating my words, "Exceptions are opportunities for extra computation."

+

+ When a geometer erases his drawing the theorem is untouched. He draws it again badly and the theorem is untouched. Every square that has ever been drawn could be wiped away and the theorem would still hold, and it held before any of them were drawn. +

-

Occasionally a student will come to me to share a great new discovery, that isn't so great, and isn't so new. Almost always the idea is not yet mature. Understandably a student's ideas are sometimes those of a person making observations while not realizing he is on a well trodden path. It is human nature to come to the conclusion that the unknown does not exist. This is not to say that professors do not harvest new ideas from students. My advice to such a student with such a discovery, is to first publish it, perhaps file a provisional patent, or at least write up and send it in an email rather than walking up and telling me about it, and then watch as the concept evolves over time and think what caused that evolution. Of course I was also a student. Hence, the Lynch-Swartlander adder patent was filed in my name (and the names of AMD design team, after all there was a cash reward for those). Also the original publication of High Radix Online Arithmetic at a small conference is in my name.

+

+ The definition of a symbol given in section is Platonic, and was written that way before the matter had been thought through. It began with a symbol, which is an abstraction; it required a representation, such as a Greek letter; and it noted that what appears on the written page is an instance of that representation, of which there can be many. Set that beside the geometers. There is the Square, there is the manner in which a square is to be presented, four sides drawn equal and joined square, and there are the many figures scratched in the sand. The symbol is the Form, the representation is the manner in which the Form is met with, and the instance is the shadow. Three levels in each case, and the middle one is the reason section declines the philosophers’ pair of type and token. +

-

Credit and showing proper respect to both students and the hard work that got professors tenure is not as easy of a thing to do as it might seem. I was once asked to present a student's work at a conference in France, and was so anxious to talk about my own work so as to attract funding, that I nearly left the summary of hers out. I remain embarrassed over this to this day. Though I am sure her colleagues properly appreciate her work. Worse mistakes than this have been made. And what could I possibly have been thinking, a small conference on applied number theory in France is more than a world apart from a Silicon Valley tech meeting.

+ -

Of course I talked to Bill to ask about the quote. I calculated that I had gained a little leverage and thought to use it to perhaps get the support for the project. His backing would help the venture pitch. Bill was obviously quite sincere when he explained that the interviewer had edited the interview to make it dramatic, and that he was himself surprised upon reading it. And of all people to offer an explanation. He could easily have instead said that he had spent most of his career using exceptions as an excuse for more computation, and he would have been right about that. I gained some respect for Bill at that time, though alas, there was no project endorsement. That might be my fault, as come to think of it, I never directly asked for it. Furthermore, nothing in that Dr. Dobbs interview discussed the content, rather it was merely the sound bite enticing the next question: what is that further computation? This book discusses this topic.

+ + Naturalism -

John Gustafson is another person who has put a lot of thought into how to formulate a valuable floating-point standard, and he now has a competing proposal. John also brought up the topic of the nature of error shortly before publishing The End of Error. He has improved and evolved that to the Posit, which abandons the variable width operands of his earlier unums in favor of a fixed width whose internal field boundaries move. This reminds me of the denorms of Std 754, but it is more graceful. Precision tapers away smoothly to the smallest representable value. John was polite and asked me about the method I had used to achieve accuracy in microcode. I answered describing High Radix Online Arithmetic and with the importance of variable precision, but I could not formulate the rest of the thoughts. I had learned a great deal while doing formal analysis of the algorithms I had put on the Am29050, Am486, and K5 processors at AMD, yet when I tried to articulate this, I could not explain it. It is articulated better in this book.

+

Aristotle (384−322 BC) arrived at Plato’s Academy at seventeen and stayed for twenty years. He was the son of a physician, and he had from the beginning the disposition of a man who wants to see the thing itself. Where Plato was brilliant and intuitive, Aristotle was scholarly and methodical. + Will Buckingham, Douglas Burnham, Peter J. King, John Marenbon, Clive Hill, and Marcus Weeks, The Philosophy Book, Big Ideas Simply Explained (New York: DK, 2011), page 58. + The difference shows in what each took to be the starting point of an inquiry.

-

Many others have given thought to the problem of errors in computation and have made effective proposals. Moore originally introduced interval methods, where two numbers bound a true value. Kulisch and Miranker attempted to formalize computer arithmetic, and started by reasoning about a screen that lay over the real number space, which is a route to the interval methods they embedded in their XSC languages. Boehm and Cartwright introduced a concept of Constructive Real Numbers, where precision of computation was extended upon demand. And through Stephen Wolfram's vision, Mathematica provides a complete symbolic math manipulation package, where a symbolic expression can be simplified. Upon a call for numeric resolution Mathematica then tracks loss of accurate precision through significance arithmetic and repeats the computation at increased precision if the accuracy of the result is not sufficient. In my own work at AMD, my approach was to perform an analysis step first, where required precision was derived, then followed by variable precision computation.

+

Aristotle rejected the Realm of Forms entirely. His objection was that it explains nothing: to account for the many circles in the world by positing one Circle elsewhere is to double the number of things standing in need of explanation while leaving the original question where it was, and to say that a drawn circle participates in the Circle is, in his words, to speak in poetical metaphors. + Aristotle, Metaphysics I.9 990b−991b, with the remark on poetical metaphors at 991a20−22, and again at XIII.4−5. Aristotle does use the word eidos, ordinarily translated as form, for the structure in virtue of which a thing is what it is. That structure is in the thing and nowhere else, and it is not Plato’s Form under another name. One English word is doing service for two concepts, and only the first is at issue here. + Take away the Realm of Forms, and nothing of the account survives. There is no elsewhere for the ideal circle to be, so there is no original for the drawn one to fall short of, so there are no shadows, so there is no cave to be led out of. The drawn circle is a circle.

-

However, all of these methods are missing a key insight. That the further computation that the exception invites us to engage in, is part of a richer second order system. The various systems mentioned above remain in the first order, though perhaps with a hint of entering a second order analysis.

+

He then turned Plato upside down. For Plato, knowledge descends: one conceptualizes the Forms, and the senses report shadows that mislead more than they inform. For Aristotle, knowledge ascends: perception of particulars is where an inquiry starts, many perceptions leave what is common to them, and the general principle is arrived at from below. + Aristotle, Posterior Analytics II.19, for knowledge beginning in perception and rising by induction to the principles. The Greek term for the operation is epagōgē, rendered as induction. +

-

In practical computing the error algebra of IEEE Std 754 can quickly mount due to the substitution flags, and the system tends towards the catch-all indeterminate value, the so-called NaN. This effect leaves some computer scientists wishing to extend the system. However this is not a failure of the system as preventing this was not the design intent. Extensions to the system do not usually fare much better, because error tag algebra can not replace computation.

+

Upon leaving the academy Aristotle then did what his position obliges a man to do, and went and looked. He dissected, he collected, and he questioned fishermen and beekeepers about what they had seen; his account of the developing chick was got by opening eggs on successive days. + Aristotle, Historia Animalium VI.3 561a for the chick; V.12 541b for the modified arm of the male octopus, which was thought a fable until confirmed in 1857. He was wrong about a great deal, holding the heart to be the seat of thought and the brain an organ for cooling the blood. Darwin, thanking William Ogle in 1882 for a translation of the Parts of Animals, wrote that Linnaeus and Cuvier had been his two gods, but that they were mere schoolboys compared to old Aristotle. Charles Darwin to William Ogle, 22 February 1882, in Francis Darwin, ed., The Life and Letters of Charles Darwin (London: John Murray, 1887), vol. 3, 252. + What he produced from it was a taxonomy: animals sorted by the features they share, the sorting answerable to the specimens and revised whenever a specimen refused its category. So Aristotle is a naturalist in both of the senses this book uses. He held that an account is owed in terms of the world, and he spent his life observing and sorting. The two are not separate facts about him. The second is what the first commits a man to.

-

With interval methods, the intervals will encompass error causing singularities that otherwise would not have manifested. However, the intervals also tend to pessimistically widen and then pull in extraneous singularities. Also, as for all these methods, exceptions can happen at run time without the code designer being aware of their nature or perhaps of the potential for it to happen. And when an exception does occur, it tends to be handled ad hoc in the manner as in conventional computation, for example by exiting the program as the interval system does not provide further guidance.

+

+ This perspective, that knowledge comes from the categorization of observations of nature rather than, as Plato taught, from the conceptualization of Forms, is the pillar of the philosophy of Naturalism. The title of this book follows from it. Computation theory is the result of observing and categorizing what the Realizable Machine of chapter does and can be made to do. +

-

The Constructive Real is reminiscent of a limit, it will lazy evaluate for increased precision until obtaining a sufficiently accurate result. A limit concept is indeed key when moving from first order computation to a second order, but such a second order system is not built in. As for interval methods, an error could surprise an algorithm designer when it appears at run time, and thus the code might not be prepared to handle it. However, there is an interesting aspect here, the expression syntax trees are available to the exception handler, or at least could be, so a layer could be built on top.

+ -

Mathematica uses symbolic manipulation of expressions combined with significance arithmetic, so numeric resolution can fail, and then the user is left holding the error as a result. However, similarly to how the Constructive Real system has syntax trees, the exception handler can have access to the unresolved symbolic expression, which the user could perform more computation with if he knew what to do with it. This is interesting.

-

There is something more fundamental going on here that all of these approaches are dancing around. There are limits to what is possible with computing and even logic itself. Attaching number representations to the structures does not change this fact. Hence, this book will start not with numerical analysis, but with the foundation of mathematics and computation theory.

+ + The word and its practitioners -

There is a first edition of this book.Thomas Walker Lynch, Tom's Turing Complete Architecture: Volume 1, Foundation and TM Library Interface (Independently published, April 24, 2023), 230 pp., ISBN 979-8392389087. The Lisp library described in that edition was released on Quicklisp. The same concepts are described in those pages, but the story is not as well told, and no one has come full circle to say that it made sense. One colleague who reviewed it admitted he was bewildered by it, and asked whether I was proposing that we build Turing Machines. After all, we are the music makers,Arthur O'Shaughnessy, "Ode", in Music and Moonlight (London: Chatto and Windus, 1874). so to speak; and we write notes in transistors and layout. So then, my colleague Kelvin rightly asked, "What is the proposal here?" I have yet to give him an answer. Perhaps this second edition will help.

+

+ Naturalism is the position that whatever there is, is natural, and that an account of anything is owed in terms of the world rather than in terms of something standing outside it. Its method follows from the position: an inquiry begins in observation of what is there. Where the two are separated, the first is called metaphysical naturalism and the second methodological naturalism, and a person can hold the second while remaining quiet about the first. +

-

So as to address Kelvin's question directly, and to prepare other readers who are familiar with my computer architecture work, let me say that there is no hardware specification for a new proposed commercial machine in this book. This book is not proposing to build a Turing Machine in hardware. This is an absolute. Building an RT Machine might be interesting. It is true that the organization for it is described in this book. Yet this is certainly not the proposal here. Rather this book explains and justifies a set of features that are planned to be incorporated into a commercial machine, and yes, that is the machine I was pitching for funding to develop in 2005.

+

+ A naturalist as a noun is one who observes and sorts. John Ray (1627−1705), Carl Linnaeus (1707−1778), the Comte de Buffon (1707−1788), and Charles Darwin (1809−1882) were naturalists in this sense, and what a naturalist produces is a taxonomy: a classification answerable to the specimens, revised whenever a specimen refuses one of its categories. Nothing in that method requires the categories to have existed before the observing. The two uses of the word are related and are not identical, and both are wanted in these pages. +

-

Though a practical commercial processor design is not provided in these pages, there is a saving grace, i.e. a practical contribution. There is a formally grounded iterator library described and available for download from GitHub. This library appeared in the first edition implemented in Lisp (SBCL, Common Lisp, rather than Scheme) and released on Quicklisp. That library is described in an appendix. Lisp is in a sense pure and formal statements can be made with it, so writing the library in Lisp made sense. Certain improvements have been made since the first edition, and this volume introduces and discusses the Python version.

+

+ It is worth recalling how recently the vocabulary shifted. The study of the physical world was called natural philosophy until the nineteenth century, and Newton’s book of 1687 is the Mathematical Principles of Natural Philosophy. The word scientist was coined in 1834.William Whewell coined it in an unsigned review of Mary Somerville, On the Connexion of the Physical Sciences, Quarterly Review 51 (1834): 54−68, and put it into print under his own name in The Philosophy of the Inductive Sciences (London: Parker, 1840), vol. 1, cxiii. The older term survives in the chairs of natural philosophy at the Scottish universities. A seventeenth century author who called mathematics a part of natural philosophy was making a claim that his vocabulary made easy, and the vocabulary changed before the claim was settled. +

+ + + Number as a natural object - +

+ The Aristotelian account was the working position of the schools for most of two thousand years, and it was still the working position when the mathematics that mattered began to change. The sixteenth century argued at length over whether mathematical demonstrations are demonstrations in the full sense, the quaestio de certitudine mathematicarum, and the argument turned on whether a mathematical object has a cause, which an Aristotelian object must.Opened by Alessandro Piccolomini, Commentarium de certitudine mathematicarum disciplinarum (Rome, 1547), and continued by Pereira, Barozzi, and Biancani into the seventeenth century. See Paolo Mancosu, Philosophy of Mathematics and Mathematical Practice in the Seventeenth Century (Oxford: Oxford University Press, 1996), chap. 1. +

- - - Introduction +

+ In the seventeenth century the naturalist reading was held plainly and by the people doing the work. Thomas Hobbes (1588−1679) made geometry a science of bodies and of the motions that generate them, and defended the position through a public quarrel with John Wallis that ran for over twenty years.Thomas Hobbes, De Corpore (London, 1655), Part II. On the dispute with Wallis, see Douglas M. Jesseph, Squaring the Circle: The War between Hobbes and Wallis (Chicago: University of Chicago Press, 1999). Isaac Barrow (1630−1677) argued in his Cambridge lectures that mathematical magnitudes are abstracted from the magnitudes of physical bodies and have no other source.Isaac Barrow, Lectiones Mathematicae, delivered 1664−1666 and published (London, 1683), Lectures I−VII. Barrow held the Lucasian chair before Newton, who succeeded him in 1669. And Newton (1642−1727), in the preface to the Principia, wrote that geometry is founded in mechanical practice and is nothing other than the part of universal mechanics which measures accurately.Isaac Newton, Philosophiæ Naturalis Principia Mathematica (London, 1687), Author’s Preface. The remark is made in passing, as something the reader will grant, which is itself evidence of how the matter then stood. A man who says that geometry is a branch of mechanics is saying that the forms are in the machinery. +

-

Kleene's Introduction to Metamathematics opens with, "A flock of four sheep and a grove of four trees are related to each other in a way in which neither is related to a pile of three stones or a grove of seven trees. ... Without counting the sheep or the trees, one can pair them with each other, for example by tethering the sheep to the trees, so that each sheep and each tree belongs to exactly one of the pairs."Stephen Cole Kleene, Introduction to Metamathematics (Amsterdam: North-Holland, 1952), 3.

+

+ The eighteenth century turned against the position from two directions at once. George Berkeley (1685−1753) denied that there are abstract general ideas at all, and then turned that denial on the calculus in The Analyst of 1734, where the vanishing increments of the new analysis are asked what they are and found to be neither finite quantities, nor nothing, but the ghosts of departed quantities.George Berkeley, A Treatise Concerning the Principles of Human Knowledge (Dublin, 1710), Introduction §§7−25, against abstract ideas; The Analyst (London, 1734), §35 for the phrase. Berkeley’s target was the reasoning and not the results, and the objection stood unanswered until the rigorization of the following century. The attack was fatal to complacency and did not restore the naturalist account, since Berkeley’s own position placed the objects in the mind rather than in the world. David Hume (1711−1776) then divided all inquiry into relations of ideas and matters of fact, and placed mathematics squarely among the former, where nothing observed bears on it.David Hume, An Enquiry Concerning Human Understanding (London, 1748), §IV part 1. The division is anticipated in A Treatise of Human Nature (London, 1739), Book I. An empiricist had thus removed mathematics from the reach of experience. +

+ -

Kleene's words are more than an introduction to Cantor's transfinite numbers. They appeal to Naturalism as a foundation. They leverage the reader's intuition, while succinctly making the implicit point that mathematics has purpose.

+ + The turn inward -

In his foundational paper Alan Turing asks us to think of a clerk sitting at a desk writing on paper while following some rules, and to imagine what can be written. Turing's own founding image is thus an observation of the world rather than an independently standing abstract form.

+

+ Immanuel Kant (1724−1804) settled the matter for the next century in the Critique of Pure Reason of 1781. Mathematical judgements are necessary, so they are not got from experience; they are also not empty, since seven and five taken together yield a twelve that no analysis of the concepts of seven, five, and sum will produce. They are therefore synthetic and a priori both, and what makes them possible is the pure intuition of space and time, which is a form contributed by the knowing subject rather than a feature found in the world.Immanuel Kant, Kritik der reinen Vernunft (Riga: Hartknoch, 1781; second edition 1787), Introduction B14−B17 for the arithmetical example, and the Transcendental Aesthetic for space and time as forms of intuition. +

+ +

+ This is the decisive turn, and it is worth being exact about what it does. Kant does not deny that mathematics is grounded. He relocates the ground, from the world to the structure of the mind that apprehends the world, and in doing so he answers the question the naturalist had been trying to answer, and answers it better than the naturalist then could. Geometry is necessary because space is our form of outer intuition, and it applies to everything we can experience because we can experience nothing outside that form. For a hundred and thirty years the position looked unassailable. +

+ +

+ What unmade it was not an argument but a discovery. Nikolai Lobachevsky (1792−1856) published a geometry in which the parallel postulate fails, in 1829; János Bolyai (1802−1860) published another in 1832; Carl Friedrich Gauss (1777−1855) had reached the same results earlier and withheld them. Bernhard Riemann (1826−1866) generalized the question in his habilitation lecture of 1854, and Eugenio Beltrami (1835−1900), whose model is discussed in chapter , showed in 1868 that the new geometry is consistent if the old one is.N. I. Lobachevsky, "On the Principles of Geometry," Kazan Messenger, 1829−1830; János Bolyai, appendix to Farkas Bolyai, Tentamen (Maros-Vásárhely, 1832); Bernhard Riemann, "Über die Hypothesen, welche der Geometrie zu Grunde liegen," delivered 1854, published Abhandlungen der Königlichen Gesellschaft der Wissenschaften zu Göttingen 13 (1868): 133−152; Eugenio Beltrami, "Saggio di interpretazione della geometria non-euclidea," Giornale di Matematiche 6 (1868): 284−312. If there are several consistent geometries and physical space satisfies at most one of them, then geometry is not the science of space, and the necessity Kant had explained is not there to be explained. The best example either party had of mathematics answering to the world had been taken off the table, and it was taken off by mathematicians rather than by philosophers. +

+ + + + The last stand and the demolition + +

+ The fullest naturalist account of number ever written appeared in the middle of this, and is therefore later than most readers expect. John Stuart Mill (1806−1873) argued in A System of Logic of 1843 that the truths of arithmetic are inductive generalizations from experience, of the same kind and standing as the generalizations of any other science. Two and one make three is a fact about collections of objects, learned the way facts about collections of objects are learned, and it is necessary only in the sense that nothing has ever contradicted it.John Stuart Mill, A System of Logic, Ratiocinative and Inductive (London: Parker, 1843), Book II, chaps. 5−6, and Book III, chap. 24. +

+ +

+ Gottlob Frege (1848−1925) destroyed the position in the Grundlagen der Arithmetik of 1884, and did so at leisure and by name. If a number is a property of a heap of things, then a single pack of cards is one, and fifty-two, and four, depending on what one has chosen to count, so the number is not in the heap. If arithmetic rests on what has been observed, then its necessity is the necessity of a habit, and a sufficiently strange experience would revise it. And if numbers are ideas, then they are episodes in somebody's head, and my two and your two are two different objects, and nothing has been said about number at all. This last charge Frege named psychologism, and it became the period's term of abuse.Gottlob Frege, Die Grundlagen der Arithmetik (Breslau: Koebner, 1884), Introduction and §§7−10 against Mill, §§21−25 against number as a property of external things, and §§26−27 against number as an idea. Frege pressed the charge again in his review of Husserl, Zeitschrift für Philosophie und philosophische Kritik 103 (1894): 313−332, with enough effect that Husserl abandoned the position. +

+ +

+ The demolition was thorough and it was, on its own terms, correct. Every one of those objections is answered in this book, and none of them is answered by disputing what Frege said about Mill. They are answered by giving numbers a source that Mill did not have available to him, which is the subject of chapter , and by an account of what a symbol is that makes my two and your two the same object, which is the subject of chapter . Mill was arguing from heaps of pebbles. There was no other mechanism then to argue from. +

+ +

+ After 1884 the field belonged to programmes that dispensed with the world. Logicism derived mathematics from logic; formalism treated it as the manipulation of marks under stated rules; intuitionism grounded it in mental construction. Richard Dedekind (1831−1916) wrote in 1888 that numbers are free creations of the human mind, and the remark drew no objection.Richard Dedekind, Was sind und was sollen die Zahlen? (Braunschweig: Vieweg, 1888), preface. Leopold Kronecker’s better-known remark that God made the whole numbers and all else is the work of man is reported by Heinrich Weber, "Leopold Kronecker," Jahresbericht der Deutschen Mathematiker-Vereinigung 2 (1893): 5−31, at 19. Formalism in particular gained by not looking at the world, since a formal system is answerable to its own rules alone and is very much the easier to study for it. The crisis recounted in chapter was fought entirely among these three, and no party to it proposed consulting an apparatus. +

+ +

+ It is therefore worth noticing what the paper that closed the episode opens with. Alan Turing, asked to settle a question in the foundations of logic, begins by describing a man sitting at a desk with paper and a pencil, and asks what such a man can be got to do. +

+ -

Here the Naturalist runs into a problem. The Turing Machine is not practical to build. The infinite tape is of course an issue, but it is perhaps not the most serious obstacle to realizing a Turing Machine. As I will formalize later in this book, the Turing Machine head must always read and react, and this will force the author of the controller to build memory into the control. In turn this causes the controller to explode in size with machine word width.

+ + The return + +

+ Naturalism came back into philosophy in the second half of the twentieth century, though not into the foundations of mathematics in any form that touched practice. Willard Van Orman Quine (1908−2000) held that epistemology is a chapter of natural science rather than a tribunal sitting above it, and that our commitment to mathematical objects stands or falls with the theories that cannot be stated without them.W. V. Quine, "Epistemology Naturalized," in Ontological Relativity and Other Essays (New York: Columbia University Press, 1969), 69−90; the indispensability argument is developed in Word and Object (Cambridge: MIT Press, 1960) and taken up by Hilary Putnam, Philosophy of Logic (New York: Harper and Row, 1971). Imre Lakatos (1922−1974) described mathematics as it is actually done, by conjecture, counterexample, and repair, which is a naturalism about the practice if not about the objects.Imre Lakatos, Proofs and Refutations: The Logic of Mathematical Discovery, ed. John Worrall and Elie Zahar (Cambridge: Cambridge University Press, 1976), from papers of 1963−1964. Hilary Putnam (1926−2016) gave the name quasi-empiricism to the view that mathematical statements are confirmed and overturned in something like the way scientific ones are.Hilary Putnam, "What is Mathematical Truth?" Historia Mathematica 2 (1975): 529−543. Philip Kitcher gave an account of mathematical knowledge as knowledge of operations that an idealized agent performs, collecting, ordering, and matching, rather than of objects standing apart.Philip Kitcher, The Nature of Mathematical Knowledge (New York: Oxford University Press, 1983). Penelope Maddy has argued the case at book length under the name second philosophy, the position that there is no vantage point above the sciences from which to correct them.Penelope Maddy, Realism in Mathematics (Oxford: Clarendon Press, 1990); Naturalism in Mathematics (Oxford: Clarendon Press, 1997); Second Philosophy: A Naturalistic Method (Oxford: Oxford University Press, 2007). +

+ +

+ Kitcher’s agent is the closest of these to the present book, and the distance is still the whole of the distance. His agent is idealized, which is to say stipulated, and what it can do is settled by the philosopher who describes it. The machine in these pages is not stipulated. It is built, and what it does when it runs is a question with an answer that nobody supplies. +

+ -

However, this situation is recoverable. This volume presents modifications that lead to an alternative computation theoretic object that is as expressive as a Turing Machine, and yet is practical to build. This then restores the validity of the Natural interpretation.

+ + What the word means here -

For a question to be resolved by a Turing Machine, the question must be formed as a program that when run produces a result in a finite number of steps. Practicality is an even stronger constraint than being computational. For running a program to be practical, finishing in a finite number of steps is not sufficient; it must also complete before a given deadline. In the natural world, we all operate under the constraints of being practical. However, that does not prevent us from having a practical discussion about what is theoretically possible to compute, and such discussion is found in the pages of this book.

+

+ Two things the word does not mean in this book. It does not mean natural computing, the field that draws its methods from living systems, from molecules, swarms, and neurons. The machine in these pages is a tape and a controller and nothing about it is borrowed from anything alive. And Natural does not qualify a number. What mathematics calls the natural numbers are Peano Numbers here, for the reason given in section , which leaves the adjective to the philosophy alone. +

-

It is commonly thought that Turing Machines, and thus observations of nature, are limited to discrete computation, and thus there is a large domain of continuous mathematics that they can not speak about. However, the very concept of the continuous is described in discrete terms. Consider the simple function x^2 over a Real field. It is considered to be a continuous function, yet it consists of discrete symbols. To take its derivative is to follow the steps of an algorithm, i.e. a program, which brings the 2 down as a multiplier and replaces the exponent with a 1, or leaves it out altogether. These are discrete steps resulting in another discrete formula, 2 x. In that the word continuous is given meaning, it becomes discrete.

+

+ What it does mean is the position reviewed above, held about mathematics, and pursued by the method the older sense of the word names. Computational Naturalism is the thesis that mathematics is a taxonomy of observations of a machine. The conditions such a machine must satisfy are set out in chapter , the machine itself is built in chapter , and the taxonomy occupies most of what follows that. +

-

Analysis is the bridge that leads from the natural world to the world of forms. Here analysis is the activity of asking about the properties of a program instead of running it. The program being analyzed then becomes the input data for the program doing the analysis. Turing's original paper introduced the naturalist interpretation of mathematics via the clerk at the desk, described a machine for automating the clerk's work, presented the idea of a stored program, described analysis, and pointed out that it was possible to analyze an analyzer. And as though that was not enough, he provided a general analysis applicable to halting analyzers that proved that no universal halting analyzer can exist. (A halting analyzer is an analyzer, i.e. a program that analyzes a given program to decide if that given program halts or not.) This does not mean that a halting analyzer can not ever be practical. For a large class of programs a halting analyzer can give an answer, and do so before a deadline. However, the proof result means that for any halting analyzer, there will be programs for which it will not be able to decide whether they halt or not, even when there is no deadline apart from 'finite number of steps'.

+

+ A reader who knows the literature will by now expect the thesis to be set against Plato, since that is how the opposition has been drawn since Aristotle drew it, and since a naturalist is supposed to be someone who denies that there is anywhere else for a number to be. This book does not draw it that way, and the remainder of this section says why, in advance of the machinery that makes the point properly. +

-

Here then, is an interesting thing about some Turing Machine programs that do not halt. Though they can not be run to completion, they can be analyzed. Say a program prints the 's' character, then loops back, and thus prints the 's' character again and again. Such a program can not be run to completion as it never halts. However, the program can be analyzed and talked about, as of course we already know, as we are doing this very thing right now. Thus the holes in the lower level language kick the problem up to a higher level where questions are asked as to how the program behaves and what an exceptional condition means.

+

+ Consider a Turing Machine program that prints the character s, loops back, and prints it again without end. It cannot be run to completion. It can nonetheless be analyzed, discussed, and reasoned about, as we are doing at this moment. Here then is the situation in miniature: there are things the first order cannot reach, and a language standing above it in which those very things are said. +

-

Accordingly the world of forms is not a separate realm. It is the tower of languages standing above the machine, each layer of it there to speak of the holes in the layer beneath. Those languages become separated from the machine that gave rise to them, so the base machine never need be run. What builds the tower is exhaustion. Each layer runs out of what it can say, and the running out is what makes the next layer necessary.

+

+ Accordingly the realm of forms is not a separate realm. It is the tower of languages standing above the machine, each layer of it there to speak of the holes in the layer beneath. Those languages become separated from the machine that gave rise to them, so the base machine never need be run. What builds the tower is exhaustion. Each layer runs out of what it can say, and the running out is what makes the next layer necessary. The layers are the orders of analysis, defined in section . +

-

The first exhaustion of this kind was not met in computing. It was met in mathematics, decades before there was a machine to describe it, and the Turing Machine is what the search for a higher view produced.

+

+ Both parties are therefore granted what they asked for. Aristotle is granted that the account begins in an object one can point at and that nothing enters from outside it; the first order is the machine running, and it is where every term in this book is finally cashed. Plato is granted a realm of forms that is genuinely above the particulars, whose inhabitants are not to be found by running anything, and which is reached by reasoning rather than by looking. What is denied is only the separation. The tower has a floor, the floor is an apparatus, and the orders are how one climbs. Neither party is the foil of the other; they are describing the same structure from opposite ends of it. +

+

+ Stephen Kleene opens his Introduction to Metamathematics with a flock of four sheep and a grove of four trees, and observes that one can pair them off, a sheep to a tree, without counting either.Stephen Cole Kleene, Introduction to Metamathematics (Amsterdam: North-Holland, 1952), 3. The passage is there to introduce Cantor, and it does something else on the way. It appeals to Naturalism as a foundation, it leverages the reader’s intuition, and it makes the implicit point that mathematics has a purpose. Nobody objects to the sheep. What this book proposes is that the appeal be taken seriously rather than used and set aside, and that the flock be replaced by an apparatus whose behaviour can be established rather than assumed. +

+ + The search that led to the Turing Machine

- The survey material here is limited to that which is relevant to the Computation Naturalism thesis of this book. It would have been easy to keep pulling on these threads and end up with a history book instead. There is far more to the story of the foundation of mathematics than what is presented here. -

- -

- In 1893 Gottlob Frege published an axiomatic construction of mathematics from set theory. Frege's grand objective was the programme later called Logicism, the philosophical thesis that all of mathematics can be derived entirely from pure logic. To bridge set theory and logic, Frege defined sets using a method known as set comprehension. Under this approach, a mathematician states a logical rule or property, and any object satisfying that logical statement automatically becomes a member of the set. Because the membership of a set is determined entirely by logical rules, the resulting sets, and the mathematics built upon them, are derived directly from logic. To implement this, his specific machinery relied upon unrestricted set comprehension, formalized as Basic Law V Gottlob Frege, Grundgesetze der Arithmetik, Vol. 1 (Jena: Hermann Pohle, 1893), §20.. + In 1893 Gottlob Frege published an axiomatic construction of mathematics from set theory. Frege’s grand objective was the programme later called Logicism, the philosophical thesis that all of mathematics can be derived entirely from pure logic. To bridge set theory and logic, Frege defined sets using a method known as set comprehension. Under this approach, a mathematician states a logical rule or property, and any object satisfying that logical statement automatically becomes a member of the set. Because the membership of a set is determined entirely by logical rules, the resulting sets, and the mathematics built upon them, are derived directly from logic. To implement this, his specific machinery relied upon unrestricted set comprehension, formalized as Basic Law V Gottlob Frege, Grundgesetze der Arithmetik, Vol. 1 (Jena: Hermann Pohle, 1893), §20..

At a conference in Paris in 1900, David Hilbert presented a list of pressing unsolved problems in mathematics. - Second on his list was "The Compatibility of the Arithmetical Axioms." Hilbert challenged mathematicians to find a means to demonstrate that "a finite number of logical steps based upon them [axioms] can never lead to contradictory results" David Hilbert, "Mathematical Problems," trans. Mary Winston Newson, Bulletin of the American Mathematical Society 8 (1902): 437–479. This can be found at https://www.gutenberg.org/cache/epub/71655/pg71655-images.html. The MathWorld article on this subject, https://mathworld.wolfram.com/HilbertsProblems.html, explains that Hilbert presented 10 problems at the conference, though the publication shows 23 problems, and a 24th problem was later discovered among Hilbert's unpublished notes. Based on the notes of this citation, it appears the second problem is the same on all of these lists. Also note, Hilbert discusses completeness specifically as an axiom for bounding on the sets, which appears to be distinct from the question of logical completeness for an axiomatic system.. + Second on his list was "The Compatibility of the Arithmetical Axioms." Hilbert challenged mathematicians to find a means to demonstrate that "a finite number of logical steps based upon them [axioms] can never lead to contradictory results" David Hilbert, "Mathematical Problems," trans. Mary Winston Newson, Bulletin of the American Mathematical Society 8 (1902): 437–479. This can be found at https://www.gutenberg.org/cache/epub/71655/pg71655-images.html. The MathWorld article on this subject, https://mathworld.wolfram.com/HilbertsProblems.html, explains that Hilbert presented 10 problems at the conference, though the publication shows 23 problems, and a 24th problem was later discovered among Hilbert’s unpublished notes. Based on the notes of this citation, it appears the second problem is the same on all of these lists. Also note, Hilbert discusses completeness specifically as an axiom for bounding on the sets, which appears to be distinct from the question of logical completeness for an axiomatic system..

- In 1901 Bertrand Russell found a well-formed set formulation using Frege's set theory that did not correspond to a set. As Frege's work was based on this set theory, this called into question his entire work. Russell pointed out that it was possible to define a set of all sets that do not contain themselves. However this was a paradox, because if said set contained itself, it shouldn't, and if it didn't it should. Thus the formulation fails to define a set because the logical condition cannot be satisfied Bertrand Russell, The Principles of Mathematics (Cambridge: Cambridge University Press, 1903), Chapter X, 'The Contradiction'.. Russell communicated this to Frege in a letter dated 1902-06-16, shortly before Frege's second volume was going to print Bertrand Russell to Gottlob Frege, June 16, 1902, reprinted in Jean van Heijenoort, From Frege to Gödel: A Source Book in Mathematical Logic (Cambridge: Harvard University Press, 1967), 124–125. Gottlob Frege, Grundgesetze der Arithmetik, Vol. 2 (Jena: Hermann Pohle, 1903), Appendix (Nachwort), 253. Frege writes: 'Hardly anything more unfortunate can befall a scientific writer than to have one of the foundations of his edifice shaken after the work is finished.'. Frege hurriedly authored an appendix (the Nachwort) admitting his system was compromised Frege was a quiet, rigid man who had spent decades building his logical fortress in almost total academic obscurity. Frege was personally devastated by Russell's letter. Shortly after, he suffered the loss of his wife, fell into severe depression, and his academic output almost entirely ceased. In 1924, a year before his death, he wrote unpublished diaries explicitly surrendering his life's work, declaring that logicism was a mistake and that mathematics must actually be derived from geometry. See I. Grattan-Guinness, The Search for Mathematical Roots, 1870–1940 (Princeton: Princeton University Press, 2000). For an analysis of Frege's intellectual decline, personal tragedies, and his unpublished 1924–1925 diaries where he formally surrenders the logicist program, see Chapter 7.. + In 1901 Bertrand Russell found a well-formed set formulation using Frege’s set theory that did not correspond to a set. As Frege’s work was based on this set theory, this called into question his entire work. Russell pointed out that it was possible to define a set of all sets that do not contain themselves. However this was a paradox, because if said set contained itself, it shouldn’t, and if it didn’t it should. Thus the formulation fails to define a set because the logical condition cannot be satisfied Bertrand Russell, The Principles of Mathematics (Cambridge: Cambridge University Press, 1903), Chapter X, ‘The Contradiction’.. Russell communicated this to Frege in a letter dated 1902-06-16, shortly before Frege’s second volume was going to print Bertrand Russell to Gottlob Frege, June 16, 1902, reprinted in Jean van Heijenoort, From Frege to Gödel: A Source Book in Mathematical Logic (Cambridge: Harvard University Press, 1967), 124–125. Gottlob Frege, Grundgesetze der Arithmetik, Vol. 2 (Jena: Hermann Pohle, 1903), Appendix (Nachwort), 253. Frege writes: ‘Hardly anything more unfortunate can befall a scientific writer than to have one of the foundations of his edifice shaken after the work is finished.’. Frege hurriedly authored an appendix (the Nachwort) admitting his system was compromised Frege was a quiet, rigid man who had spent decades building his logical fortress in almost total academic obscurity. Frege was personally devastated by Russell’s letter. Shortly after, he suffered the loss of his wife, fell into severe depression, and his academic output almost entirely ceased. In 1924, a year before his death, he wrote unpublished diaries explicitly surrendering his life’s work, declaring that logicism was a mistake and that mathematics must actually be derived from geometry. See I. Grattan-Guinness, The Search for Mathematical Roots, 1870–1940 (Princeton: Princeton University Press, 2000). For an analysis of Frege’s intellectual decline, personal tragedies, and his unpublished 1924–1925 diaries where he formally surrenders the logicist program, see Chapter 7..

- In 1903 Russell proposed a hierarchy of types to repair this foundational vulnerability. At the base were sets of individuals, then sets based on individuals or sets of individuals, etc. This looks a lot like how types work in modern software Bertrand Russell, The Principles of Mathematics (Cambridge: Cambridge University Press, 1903), Appendix B: The Doctrine of Types.. In this manner, it is not possible to write a paradoxical set definition. Russell and Alfred North Whitehead then engineered an entirely new, massive scaffolding utilizing this type system to pursue Frege's original objective of deriving mathematics from logic, publishing their results in three volumes between 1910 and 1913 Alfred North Whitehead and Bertrand Russell, Principia Mathematica (Cambridge: Cambridge University Press, 1910–1913).. Russell's system can be cumbersome due to requiring a large construction to be assembled in place of what otherwise might have been a simple rule. + In 1903 Russell proposed a hierarchy of types to repair this foundational vulnerability. At the base were sets of individuals, then sets based on individuals or sets of individuals, etc. This looks a lot like how types work in modern software Bertrand Russell, The Principles of Mathematics (Cambridge: Cambridge University Press, 1903), Appendix B: The Doctrine of Types.. In this manner, it is not possible to write a paradoxical set definition. Russell and Alfred North Whitehead then engineered an entirely new, massive scaffolding utilizing this type system to pursue Frege’s original objective of deriving mathematics from logic, publishing their results in three volumes between 1910 and 1913 Alfred North Whitehead and Bertrand Russell, Principia Mathematica (Cambridge: Cambridge University Press, 1910–1913).. Russell’s system can be cumbersome due to requiring a large construction to be assembled in place of what otherwise might have been a simple rule.

- In 1908 Ernst Zermelo published an alternative system designed to avoid the known paradoxical statements of the time, even though absolute consistency remained unproven. In Zermelo's set theory, a mathematician first starts with an existing set, and then applies the Axiom of Separation using definite properties to partition out subsetsErnst Zermelo, "Untersuchungen über die Grundlagen der Mengenlehre I," Mathematische Annalen 65 (1908): 261–281.. + In 1908 Ernst Zermelo published an alternative system designed to avoid the known paradoxical statements of the time, even though absolute consistency remained unproven. In Zermelo’s set theory, a mathematician first starts with an existing set, and then applies the Axiom of Separation using definite properties to partition out subsetsErnst Zermelo, "Untersuchungen über die Grundlagen der Mengenlehre I," Mathematische Annalen 65 (1908): 261–281..

- To see how this works, begin with the unrestricted case. A logician is permitted to define the predicate P(x) as x ∉ x. Then \{x \mid P(x)\} produces Russell's paradox, so the set fails to be defined. This is inconsistent with the founding assumption that any predicate would work, so it is a problem. In contrast, consider the same condition, though restricted by Zermelo's Axiom of Separation over a predefined set S, written as \dot{R} = \{x \mid x ∈ S ∧ x ∉ x\}. Now the predicate has two terms. The only thing a person needs to know about S here is that it has already been successfully defined; we don't need to know what that definition is. Now assume \dot{R} is in S. That gives the second term of the condition authority, which enables the familiar fatal loop: if \dot{R} is in \dot{R}, it shouldn't be, and if it isn't, it should be. And thus it is clear that the initial assumption, that \dot{R} is in S, must be wrong, and \dot{R} is not in S. Authority returns to the first term, x ∈ S, which is false, so the condition is false. No contradiction follows, and the paradox vanishes. + To see how this works, begin with the unrestricted case. A logician is permitted to define the predicate P(x) as x ∉ x. Then \{x \mid P(x)\} produces Russell’s paradox, so the set fails to be defined. This is inconsistent with the founding assumption that any predicate would work, so it is a problem. In contrast, consider the same condition, though restricted by Zermelo’s Axiom of Separation over a predefined set S, written as \dot{R} = \{x \mid x ∈ S ∧ x ∉ x\}. Now the predicate has two terms. The only thing a person needs to know about S here is that it has already been successfully defined; we don’t need to know what that definition is. Now assume \dot{R} is in S. That gives the second term of the condition authority, which enables the familiar fatal loop: if \dot{R} is in \dot{R}, it shouldn’t be, and if it isn’t, it should be. And thus it is clear that the initial assumption, that \dot{R} is in S, must be wrong, and \dot{R} is not in S. Authority returns to the first term, x ∈ S, which is false, so the condition is false. No contradiction follows, and the paradox vanishes.

@@ -456,7 +590,7 @@

- The authority to remove Russell's Paradox set formulation comes from the set S. It is not that undefined sets are disallowed, as \dot{R} is perfectly well defined, the Axiom of Separation having defined it. It is that membership in S is granted by prior construction and never by satisfying a condition. A condition can only partition what S already holds, so it has no power to admit. This is why the very condition that is fatal under unrestricted comprehension is harmless under separation: in the first it was doing the admitting, and in the second it is not. Though this invites an objection. If S is stocked in advance, and a condition can only sort what is already inside, then have we not merely arranged for \dot{R} to be absent and called the arrangement a repair? I sometimes wonder how mathematics might have evolved had Frege simply taken that approach and declared such formulations dismissed. It is not the same thing, and showing why requires machinery we do not have yet. We take the question up again in section . + The authority to remove Russell’s Paradox set formulation comes from the set S. It is not that undefined sets are disallowed, as \dot{R} is perfectly well defined, the Axiom of Separation having defined it. It is that membership in S is granted by prior construction and never by satisfying a condition. A condition can only partition what S already holds, so it has no power to admit. This is why the very condition that is fatal under unrestricted comprehension is harmless under separation: in the first it was doing the admitting, and in the second it is not. Though this invites an objection. If S is stocked in advance, and a condition can only sort what is already inside, then have we not merely arranged for \dot{R} to be absent and called the arrangement a repair? I sometimes wonder how mathematics might have evolved had Frege simply taken that approach and declared such formulations dismissed. It is not the same thing, and showing why requires machinery we do not have yet. We take the question up again in section .

@@ -464,30 +598,30 @@

In 1922 Hilbert lectured on the foundations of arithmetic, and he appears to be the first to use the word metamathematics in its modern sense, for the study of formal systems as objects, in which axioms, formulas, and proofs are treated as finite arrangements of symbols and are reasoned about from outside, by means that the system under examination need not itself contain - David Hilbert, "Neubegründung der Mathematik. Erste Mitteilung," Abhandlungen aus dem Mathematischen Seminar der Universität Hamburg 1 (1922): 157–177. The word reached print as Metamathematik in David Hilbert, "Die logischen Grundlagen der Mathematik," Mathematische Annalen 88 (1923): 151–165. Hilbert did not coin the word. It appears in English in the 1870s as a term of derision for speculation about non-Euclidean geometry, formed on the model of 'metaphysics' and carrying the same charge of idle abstraction. Charles Porterfield Krauth, A Vocabulary of the Philosophical Sciences (New York: Sheldon, 1879), defines it as the philosophy of mathematics, and the 1890 Funk & Wagnalls dictionary gives 'the philosophy or metaphysics of mathematics'. What Hilbert repurposed was a slur that had been aimed at the very geometry which made the subject necessary. On the earlier usage see Jeff Miller, "Earliest Known Uses of Some of the Words of Mathematics," entry for METAMATHEMATICS, which credits the observation to John Aldrich.. - The distinction the word draws is between working within a system and working upon one. The practice predates the branding. In one of the better known examples, when Beltrami built a surface upon which Euclid's parallel postulate fails, he proved nothing about triangles. He proved that one axiom could not be derived from the others, which is a fact about a collection of axioms and not about space + David Hilbert, "Neubegründung der Mathematik. Erste Mitteilung," Abhandlungen aus dem Mathematischen Seminar der Universität Hamburg 1 (1922): 157–177. The word reached print as Metamathematik in David Hilbert, "Die logischen Grundlagen der Mathematik," Mathematische Annalen 88 (1923): 151–165. Hilbert did not coin the word. It appears in English in the 1870s as a term of derision for speculation about non-Euclidean geometry, formed on the model of ‘metaphysics’ and carrying the same charge of idle abstraction. Charles Porterfield Krauth, A Vocabulary of the Philosophical Sciences (New York: Sheldon, 1879), defines it as the philosophy of mathematics, and the 1890 Funk & Wagnalls dictionary gives ‘the philosophy or metaphysics of mathematics’. What Hilbert repurposed was a slur that had been aimed at the very geometry which made the subject necessary. On the earlier usage see Jeff Miller, "Earliest Known Uses of Some of the Words of Mathematics," entry for METAMATHEMATICS, which credits the observation to John Aldrich.. + The distinction the word draws is between working within a system and working upon one. The practice predates the branding. In one of the better known examples, when Beltrami built a surface upon which Euclid’s parallel postulate fails, he proved nothing about triangles. He proved that one axiom could not be derived from the others, which is a fact about a collection of axioms and not about space Eugenio Beltrami, "Saggio di interpretazione della geometria non-euclidea," Giornale di Matematiche 6 (1868): 284–312.. - Russell supplies another example from earlier in this chapter. When he exhibited his set, he proved nothing about sets. He proved that Frege's system was inconsistent. In each case the object under examination was a mathematical system, and the examination was itself done using mathematics. + Russell supplies another example from earlier in this chapter. When he exhibited his set, he proved nothing about sets. He proved that Frege’s system was inconsistent. In each case the object under examination was a mathematical system, and the examination was itself done using mathematics.

In 1928 David Hilbert and Wilhelm Ackermann published a textbook on mathematical logic, Grundzüge der theoretischen Logik David Hilbert and Wilhelm Ackermann, Grundzüge der theoretischen Logik (Berlin: Springer, 1928). This first edition has not been translated into English.. A feature of this book is its attention to procedures to follow for mechanically determining truth of statements. They called the problem solved by such a procedure the Entscheidungsproblem - The term Entscheidungsproblem literally translates to 'decision problem'. However, there are many types of decision problems, and later we will meet a class of Turing Machine programs called deciders, so it appears to be best to keep the original German. As we will see later Alan Turing also did this.. + The term Entscheidungsproblem literally translates to ‘decision problem’. However, there are many types of decision problems, and later we will meet a class of Turing Machine programs called deciders, so it appears to be best to keep the original German. As we will see later Alan Turing also did this.. In the first chapter they review the procedure that solves it for the propositional logic. Raising the same question for the first-order predicate calculus, they ask the question of metamathematics: how one can determine, for an arbitrary logical expression, whether it represents a true assertion under every substitution for the variables occurring in it - Hilbert and Ackermann, Grundzüge der theoretischen Logik, 72–73. The authors' formulation restricts the question to expressions containing no individual signs, that is, no constants. This is a normalisation of the input rather than a limitation on what can be asked. A constant can be replaced by a fresh variable without affecting the answer, and the authors do this on page 73 for propositional variables, noting that these too can always be eliminated. Where a system of axioms fixes the meaning of its predicates, as with the geometry worked through on pages 74 to 76, those predicates are likewise replaced by variables and the content of the axioms is gathered into the antecedent of a single implication, so that a question about a particular subject matter becomes a question about the validity of a formula containing nothing determinate at all. I have left the restriction out of the sentence above because it answers a question the reader has not yet had occasion to ask.. + Hilbert and Ackermann, Grundzüge der theoretischen Logik, 72–73. The authors’ formulation restricts the question to expressions containing no individual signs, that is, no constants. This is a normalisation of the input rather than a limitation on what can be asked. A constant can be replaced by a fresh variable without affecting the answer, and the authors do this on page 73 for propositional variables, noting that these too can always be eliminated. Where a system of axioms fixes the meaning of its predicates, as with the geometry worked through on pages 74 to 76, those predicates are likewise replaced by variables and the content of the axioms is gathered into the antecedent of a single implication, so that a question about a particular subject matter becomes a question about the validity of a formula containing nothing determinate at all. I have left the restriction out of the sentence above because it answers a question the reader has not yet had occasion to ask.. They then show what such a procedure would be worth, working through the axioms of geometry: it would mechanically settle whether a given theorem follows from a given system of axioms, and whether that system is consistent at all. Having reviewed the special cases that have been solved, including one by Ackermann himself, then still awaiting publication, they concede that "A general solution to the Entscheidungsproblem, whether one takes the first or the second formulation, is not yet available." Hilbert and Ackermann, Grundzüge der theoretischen Logik, 81. "Eine allgemeine Lösung des Entscheidungsproblems, mag man nun die erste oder die zweite Fassung nehmen, liegt bis jetzt noch nicht vor." The two formulations are distinguished on page 80: the first asks, of a given expression, for which domains of individuals it is valid and for which it is not; the second asks only whether it is valid for all domains. The second suffices for deciding whether a theorem follows from a system of axioms. And so they throw down the gauntlet: "The fundamental significance of the Entscheidungsproblem should by now be sufficiently clear; the Entscheidungsproblem must be regarded as the main problem of mathematical logic." - Hilbert and Ackermann, Grundzüge der theoretischen Logik, 77. "Die fundamentale Bedeutung, die das Entscheidungsproblem besitzt, dürfte damit genügend illustriert sein; das Entscheidungsproblem muß als das Hauptproblem der mathematischen Logik bezeichnet werden." The italics are the authors' own. Note that this sentence precedes the concession quoted above: it closes § 11, on page 77, whereas the admission that no general solution exists opens the survey of solved special cases in § 12, on page 81. I have reversed the order because the declaration reads to a modern eye as a response to the open problem, which is how the following decade received it, while in the book it serves to introduce the survey that establishes the problem is open. The sentence is conventionally rendered "the decision problem must be called the main problem of mathematical logic"; my translation differs in three small ways. I keep the German Entscheidungsproblem throughout, for the reason given above. I render bezeichnet werden as "regarded as" rather than the literal "designated as", because modern English "designate" reads as an act of naming, whereas the German here is passing judgement on the problem's standing. And dürfte damit genügend illustriert sein is a courteous subjunctive with no close modern equivalent — literally "may thereby be presumed to be sufficiently illustrated" — which I have flattened to "should by now be sufficiently clear". + Hilbert and Ackermann, Grundzüge der theoretischen Logik, 77. "Die fundamentale Bedeutung, die das Entscheidungsproblem besitzt, dürfte damit genügend illustriert sein; das Entscheidungsproblem muß als das Hauptproblem der mathematischen Logik bezeichnet werden." The italics are the authors’ own. Note that this sentence precedes the concession quoted above: it closes § 11, on page 77, whereas the admission that no general solution exists opens the survey of solved special cases in § 12, on page 81. I have reversed the order because the declaration reads to a modern eye as a response to the open problem, which is how the following decade received it, while in the book it serves to introduce the survey that establishes the problem is open. The sentence is conventionally rendered "the decision problem must be called the main problem of mathematical logic"; my translation differs in three small ways. I keep the German Entscheidungsproblem throughout, for the reason given above. I render bezeichnet werden as "regarded as" rather than the literal "designated as", because modern English "designate" reads as an act of naming, whereas the German here is passing judgement on the problem’s standing. And dürfte damit genügend illustriert sein is a courteous subjunctive with no close modern equivalent — literally "may thereby be presumed to be sufficiently illustrated" — which I have flattened to "should by now be sufficiently clear".

- In 1931 Kurt Gödel published his incompleteness theorems Kurt Gödel, "Über formal unentscheidbare Sätze der Principia Mathematica und verwandter Systeme I," Monatshefte für Mathematik und Physik 38 (1931): 173–198.. By mapping formal logic into arithmetic, he demonstrated that any consistent, effectively axiomatized formal system sufficiently powerful to perform basic arithmetic, let us call it system F, will inevitably contain well-formed formulas that are mathematically true yet cannot be proven within the system itself For the definitive English translation, see Jean van Heijenoort, From Frege to Gödel: A Source Book in Mathematical Logic, 1879–1931 (Cambridge: Harvard University Press, 1967), 596–616.. Gödel achieved this by engineering a specific formula that evaluates to the claim: "G: There exists no sequence of valid logical steps within system F that proves G." If system F is consistent, it cannot output a proof for G; thus, the claim G makes is factually accurate, rendering it true but mechanically unprovable. Furthermore, Gödel demonstrated that system F cannot output a proof of its own consistency. This result fractured David Hilbert's 1900 vision of utilizing a weaker, strictly "finitistic" logical subsystem to definitively prove that the axioms of arithmetic are entirely free of contradictions David Hilbert, "Mathematical Problems," Bulletin of the American Mathematical Society 8 (1902): 437–479.. If the full, powerful system F lacks the mechanical capacity to verify its own consistency, Hilbert's weaker finitistic subsystem is definitively incapable of accomplishing the task. Gödel's work established a hard mechanical boundary, asserting that truth and provability are distinct concepts in classical mathematics. + In 1931 Kurt Gödel published his incompleteness theorems Kurt Gödel, "Über formal unentscheidbare Sätze der Principia Mathematica und verwandter Systeme I," Monatshefte für Mathematik und Physik 38 (1931): 173–198.. By mapping formal logic into arithmetic, he demonstrated that any consistent, effectively axiomatized formal system sufficiently powerful to perform basic arithmetic, let us call it system F, will inevitably contain well-formed formulas that are mathematically true yet cannot be proven within the system itself For the definitive English translation, see Jean van Heijenoort, From Frege to Gödel: A Source Book in Mathematical Logic, 1879–1931 (Cambridge: Harvard University Press, 1967), 596–616.. Gödel achieved this by engineering a specific formula that evaluates to the claim: "G: There exists no sequence of valid logical steps within system F that proves G." If system F is consistent, it cannot output a proof for G; thus, the claim G makes is factually accurate, rendering it true but mechanically unprovable. Furthermore, Gödel demonstrated that system F cannot output a proof of its own consistency. This result fractured David Hilbert’s 1900 vision of utilizing a weaker, strictly "finitistic" logical subsystem to definitively prove that the axioms of arithmetic are entirely free of contradictions David Hilbert, "Mathematical Problems," Bulletin of the American Mathematical Society 8 (1902): 437–479.. If the full, powerful system F lacks the mechanical capacity to verify its own consistency, Hilbert’s weaker finitistic subsystem is definitively incapable of accomplishing the task. Gödel’s work established a hard mechanical boundary, asserting that truth and provability are distinct concepts in classical mathematics.

- In 1936-03, Alonzo Church leveraged Gödel's foundational papers to directly answer the Entscheidungsproblem Alonzo Church, "A Note on the Entscheidungsproblem," The Journal of Symbolic Logic 1, no. 1 (March 1936): 40–41; correction, The Journal of Symbolic Logic 1, no. 3 (September 1936): 101–102., resting that argument on the unsolvability result he published the following month Alonzo Church, "An Unsolvable Problem of Elementary Number Theory," American Journal of Mathematics 58, no. 2 (April 1936): 345–363.. Working independently, Alan Turing had arrived at his own mechanical solution, and upon seeing Church's work, Turing rushed to submit his manuscript, which was received on 1936-05-28 Alan M. Turing, "On Computable Numbers, with an Application to the Entscheidungsproblem," Proceedings of the London Mathematical Society s2 42, no. 1 (1936): 230–265. Received May 28, 1936, read November 12, 1936.. On 1936-08-28 he appended a proof that his mechanical architecture was mathematically equivalent to Church's lambda calculus. + In 1936-03, Alonzo Church leveraged Gödel’s foundational papers to directly answer the Entscheidungsproblem Alonzo Church, "A Note on the Entscheidungsproblem," The Journal of Symbolic Logic 1, no. 1 (March 1936): 40–41; correction, The Journal of Symbolic Logic 1, no. 3 (September 1936): 101–102., resting that argument on the unsolvability result he published the following month Alonzo Church, "An Unsolvable Problem of Elementary Number Theory," American Journal of Mathematics 58, no. 2 (April 1936): 345–363.. Working independently, Alan Turing had arrived at his own mechanical solution, and upon seeing Church’s work, Turing rushed to submit his manuscript, which was received on 1936-05-28 Alan M. Turing, "On Computable Numbers, with an Application to the Entscheidungsproblem," Proceedings of the London Mathematical Society s2 42, no. 1 (1936): 230–265. Received May 28, 1936, read November 12, 1936.. On 1936-08-28 he appended a proof that his mechanical architecture was mathematically equivalent to Church’s lambda calculus.

@@ -496,7 +630,7 @@

- Turing employed an enumerative diagonal argument to establish this result. A simpler proof by contradiction that is commonly used today was first published by Christopher Strachey in 1965 Christopher Strachey, "An Impossible Program," The Computer Journal 7, no. 4 (January 1965): 313. In his letter, Strachey explicitly attributed the distilled logic to an existing "well known piece of folklore among programmers.". To begin the proof, assume a person builds a perfect decider program, H(p, i), that evaluates any given program p executing with input i, then outputs 'Y' if p(i) halts, and 'N' if it does not halt. Next, a person writes a malicious program, M(x), that incorporates H as a subroutine. When M receives an input program x, it evaluates H(x, x) to determine how program x behaves when given itself as input. If H(x, x) outputs 'Y', M enters an infinite loop; if H(x, x) outputs 'N', M immediately halts. + Turing employed an enumerative diagonal argument to establish this result. A simpler proof by contradiction that is commonly used today was first published by Christopher Strachey in 1965 Christopher Strachey, "An Impossible Program," The Computer Journal 7, no. 4 (January 1965): 313. In his letter, Strachey explicitly attributed the distilled logic to an existing "well known piece of folklore among programmers.". To begin the proof, assume a person builds a perfect decider program, H(p, i), that evaluates any given program p executing with input i, then outputs ‘Y’ if p(i) halts, and ‘N’ if it does not halt. Next, a person writes a malicious program, M(x), that incorporates H as a subroutine. When M receives an input program x, it evaluates H(x, x) to determine how program x behaves when given itself as input. If H(x, x) outputs ‘Y’, M enters an infinite loop; if H(x, x) outputs ‘N’, M immediately halts.

@@ -511,15 +645,15 @@

- The evil part occurs when we give program M(x) itself as input, M(M). Program M calls its subroutine and asks H(M, M) what M will do. If H outputs 'Y', it is wrong, because M loops infinitely. If H outputs 'N', then it is wrong, because M halts. The decider H is forced into an inescapable failure, proving that no universal decider can exist. + The evil part occurs when we give program M(x) itself as input, M(M). Program M calls its subroutine and asks H(M, M) what M will do. If H outputs ‘Y’, it is wrong, because M loops infinitely. If H outputs ‘N’, then it is wrong, because M halts. The decider H is forced into an inescapable failure, proving that no universal decider can exist.

- While Gödel, Church, and Turing established the primary boundaries of computation, they did not work in a vacuum. During this period, the broader academic community worked to synthesize the definitive mechanics of effective calculability. Jacques Herbrand and Gödel formalized general recursive functions between 1931 and 1934 Kurt Gödel, "On Undecidable Propositions of Formal Mathematical Systems," mimeographed lecture notes, Institute for Advanced Study, Princeton, 1934.. Emil Post independently defined "Finite Combinatory Processes" in 1936, outlining a theoretical architecture functionally identical to Turing's model Emil L. Post, "Finite Combinatory Processes—Formulation 1," The Journal of Symbolic Logic 1, no. 3 (September 1936): 103–105.. Stephen Kleene subsequently unified these disparate threads, proving the strict mathematical equivalence of Church's lambda calculus and Herbrand-Gödel recursive functions Stephen C. Kleene, "General Recursive Functions of Natural Numbers," Mathematische Annalen 112 (1936): 727–742.. Turing supplied the remaining leg himself, showing his machines equivalent to the lambda calculus Alan M. Turing, "Computability and λ-definability," The Journal of Symbolic Logic 2, no. 4 (December 1937): 153–163.. + While Gödel, Church, and Turing established the primary boundaries of computation, they did not work in a vacuum. During this period, the broader academic community worked to synthesize the definitive mechanics of effective calculability. Jacques Herbrand and Gödel formalized general recursive functions between 1931 and 1934 Kurt Gödel, "On Undecidable Propositions of Formal Mathematical Systems," mimeographed lecture notes, Institute for Advanced Study, Princeton, 1934.. Emil Post independently defined "Finite Combinatory Processes" in 1936, outlining a theoretical architecture functionally identical to Turing’s model Emil L. Post, "Finite Combinatory Processes—Formulation 1," The Journal of Symbolic Logic 1, no. 3 (September 1936): 103–105.. Stephen Kleene subsequently unified these disparate threads, proving the strict mathematical equivalence of Church’s lambda calculus and Herbrand-Gödel recursive functions Stephen C. Kleene, "General Recursive Functions of Natural Numbers," Mathematische Annalen 112 (1936): 727–742.. Turing supplied the remaining leg himself, showing his machines equivalent to the lambda calculus Alan M. Turing, "Computability and λ-definability," The Journal of Symbolic Logic 2, no. 4 (December 1937): 153–163..

- The academic community was thus equipped with three mathematically equivalent foundations for computation theory: recursive functions, the lambda calculus, and the Turing Machine. While all three frameworks remain active subjects of study, Turing's became the foundation of choice for computation theory textbooks by Stephen Kleene Stephen C. Kleene, Introduction to Metamathematics (Amsterdam: North-Holland, 1952)., Martin Davis Martin Davis, Computability and Unsolvability (New York: McGraw-Hill, 1958)., and Marvin Minsky Marvin L. Minsky, Computation: Finite and Infinite Machines (Englewood Cliffs: Prentice-Hall, 1967)., leading to the modern standard presentations by authors such as John Hopcroft and Jeffrey Ullman John E. Hopcroft and Jeffrey D. Ullman, Introduction to Automata Theory, Languages, and Computation (Reading: Addison-Wesley, 1979)., as well as Harry Lewis and Christos Papadimitriou Harry R. Lewis and Christos H. Papadimitriou, Elements of the Theory of Computation (Englewood Cliffs: Prentice-Hall, 1981).. + The academic community was thus equipped with three mathematically equivalent foundations for computation theory: recursive functions, the lambda calculus, and the Turing Machine. While all three frameworks remain active subjects of study, Turing’s became the foundation of choice for computation theory textbooks by Stephen Kleene Stephen C. Kleene, Introduction to Metamathematics (Amsterdam: North-Holland, 1952)., Martin Davis Martin Davis, Computability and Unsolvability (New York: McGraw-Hill, 1958)., and Marvin Minsky Marvin L. Minsky, Computation: Finite and Infinite Machines (Englewood Cliffs: Prentice-Hall, 1967)., leading to the modern standard presentations by authors such as John Hopcroft and Jeffrey Ullman John E. Hopcroft and Jeffrey D. Ullman, Introduction to Automata Theory, Languages, and Computation (Reading: Addison-Wesley, 1979)., as well as Harry Lewis and Christos Papadimitriou Harry R. Lewis and Christos H. Papadimitriou, Elements of the Theory of Computation (Englewood Cliffs: Prentice-Hall, 1981)..

@@ -529,37 +663,37 @@ On Turing and Babbage

- In reading Alan Turing's 1936 paper, it is striking how modern the text feels, specifically because he discusses algorithms, stored programs, and the mechanical limits of computation. While his contemporaries largely built purely mathematical and logical frameworks, Turing uniquely tied computation theory directly to the abstraction of machines executing stored programs. Because physical hardware capable of executing stored programs had not yet been invented, this explicit architectural grounding makes Turing's work remarkably prescient. So it is not only theoreticians who take an interest in the original paper. Historians of computer architecture ask a question of their own about it: how big a leap was it? There was a leap, though as this chapter argues it is not the one they are looking for. + In reading Alan Turing’s 1936 paper, it is striking how modern the text feels, specifically because he discusses algorithms, stored programs, and the mechanical limits of computation. While his contemporaries largely built purely mathematical and logical frameworks, Turing uniquely tied computation theory directly to the abstraction of machines executing stored programs. Because physical hardware capable of executing stored programs had not yet been invented, this explicit architectural grounding makes Turing’s work remarkably prescient. So it is not only theoreticians who take an interest in the original paper. Historians of computer architecture ask a question of their own about it: how big a leap was it? There was a leap, though as this chapter argues it is not the one they are looking for.

- Modern computer architecture dates back to fellow Englishman Charles Babbage's Analytical Engine of 1837. It had an arithmetic mill, a store of a thousand numbers, and control by punched card, with conditional branching and looping. Babbage had held the Lucasian chair at Cambridge, where Turing was later a Fellow of King's, and the accounts of the engine were in print and on the shelves. Menabrea's description, in the Countess of Lovelace's translation and with her notes, had been available since 1843, and Babbage's own chapter on the engine since 1864L. F. Menabrea, "Sketch of the Analytical Engine Invented by Charles Babbage," trans. Ada Augusta, Countess of Lovelace, with translator's notes, in Richard Taylor, ed., Scientific Memoirs, vol. 3 (London: Richard and John E. Taylor, 1843), 666–731; Charles Babbage, Passages from the Life of a Philosopher (London: Longman, Green, 1864), chap. VIII.. Turing's paper does not mention any of it. + Modern computer architecture dates back to fellow Englishman Charles Babbage’s Analytical Engine of 1837. It had an arithmetic mill, a store of a thousand numbers, and control by punched card, with conditional branching and looping. Babbage had held the Lucasian chair at Cambridge, where Turing was later a Fellow of King’s, and the accounts of the engine were in print and on the shelves. Menabrea’s description, in the Countess of Lovelace’s translation and with her notes, had been available since 1843, and Babbage’s own chapter on the engine since 1864L. F. Menabrea, "Sketch of the Analytical Engine Invented by Charles Babbage," trans. Ada Augusta, Countess of Lovelace, with translator’s notes, in Richard Taylor, ed., Scientific Memoirs, vol. 3 (London: Richard and John E. Taylor, 1843), 666–731; Charles Babbage, Passages from the Life of a Philosopher (London: Longman, Green, 1864), chap. VIII.. Turing’s paper does not mention any of it.

- So the question has been asked as to why Turing did not cite Babbage. Robin Gandy takes it up in his preface to the paper in Turing's collected worksR. O. Gandy, preface to "On Computable Numbers, with an Application to the Entscheidungsproblem," in R. O. Gandy and C. E. M. Yates, eds., Mathematical Logic, The Collected Works of A. M. Turing (Amsterdam: North-Holland, 2001). Gandy directs the reader to his own study of the period for the background of ideas and for the contributions of Hilbert's school, of Church and his students, and of Post: R. O. Gandy, "The Confluence of Ideas in 1936," in Rolf Herken, ed., The Universal Turing Machine: A Half-Century Survey (Oxford: Oxford University Press, 1988), 55–111.. He argues that Turing cannot have read Babbage, on the ground that he would have said so had he done so. Gandy allows that Turing might have seen the article on calculating machines in the eleventh edition of the Encyclopaedia Britannica, a copy of which Turing inherited from his father and consulted in later years, but observes that its treatment of the Analytical Engine is brief and dismissive, and would not have suggested to a reader that Babbage had conceived a universal machine. What can be said against this is that the material was there to be read. If Turing did not read these things, it was by choice or by accident, and not for want of access. + So the question has been asked as to why Turing did not cite Babbage. Robin Gandy takes it up in his preface to the paper in Turing’s collected worksR. O. Gandy, preface to "On Computable Numbers, with an Application to the Entscheidungsproblem," in R. O. Gandy and C. E. M. Yates, eds., Mathematical Logic, The Collected Works of A. M. Turing (Amsterdam: North-Holland, 2001). Gandy directs the reader to his own study of the period for the background of ideas and for the contributions of Hilbert’s school, of Church and his students, and of Post: R. O. Gandy, "The Confluence of Ideas in 1936," in Rolf Herken, ed., The Universal Turing Machine: A Half-Century Survey (Oxford: Oxford University Press, 1988), 55–111.. He argues that Turing cannot have read Babbage, on the ground that he would have said so had he done so. Gandy allows that Turing might have seen the article on calculating machines in the eleventh edition of the Encyclopaedia Britannica, a copy of which Turing inherited from his father and consulted in later years, but observes that its treatment of the Analytical Engine is brief and dismissive, and would not have suggested to a reader that Babbage had conceived a universal machine. What can be said against this is that the material was there to be read. If Turing did not read these things, it was by choice or by accident, and not for want of access.

- Gandy's reasoning is an argument from silence. It moves from an absence in the text to an absence in the reading, and nothing short of a borrowing record or a letter could tell against it. There is nothing strange in originality arising from ignorance. A person who has not read a thing may well arrive at it himself, and often does, and a student who reaches an idea for the first time has reached it whether or not others reached it before him. What is strange is the use the ignorance is put to here. The academic standard for an original contribution asks what a man did that had not been done, and asks him to acknowledge what he drew upon. It does not ordinarily accept an absence of reading as a ground for the credit. Gandy offers one. Against which stands the plain fact that Gandy knew the man, and that most of what can be said about the period at all is said in his own study of it. + Gandy’s reasoning is an argument from silence. It moves from an absence in the text to an absence in the reading, and nothing short of a borrowing record or a letter could tell against it. There is nothing strange in originality arising from ignorance. A person who has not read a thing may well arrive at it himself, and often does, and a student who reaches an idea for the first time has reached it whether or not others reached it before him. What is strange is the use the ignorance is put to here. The academic standard for an original contribution asks what a man did that had not been done, and asks him to acknowledge what he drew upon. It does not ordinarily accept an absence of reading as a ground for the credit. Gandy offers one. Against which stands the plain fact that Gandy knew the man, and that most of what can be said about the period at all is said in his own study of it.

-

Turing's paper is full of references to machines. He writes about tape and heads and configurations, about what a machine can be made to do and what it cannot be made to do. A man whose mind is running on machines to that degree is not a likely candidate for having read nothing about the topic.

+

Turing’s paper is full of references to machines. He writes about tape and heads and configurations, about what a machine can be made to do and what it cannot be made to do. A man whose mind is running on machines to that degree is not a likely candidate for having read nothing about the topic.

-

But also notice, the paper is a mathematics paper, and its citations are the handful of mathematical works the argument stands on, Gödel and Church and Hilbert and Ackermann among them, and it surveys nothing. Babbage's engine was not a source for the argument. It belonged to the common stock of what a machine was. Turing advanced no claim about computer architecture anywhere in those pages, for the sufficient reason that computer architecture was not what he was writing about.

+

But also notice, the paper is a mathematics paper, and its citations are the handful of mathematical works the argument stands on, Gödel and Church and Hilbert and Ackermann among them, and it surveys nothing. Babbage’s engine was not a source for the argument. It belonged to the common stock of what a machine was. Turing advanced no claim about computer architecture anywhere in those pages, for the sufficient reason that computer architecture was not what he was writing about.

As archaeologists of computer history, we have decided what is important in computer architecture, and what is not, and only then gone back to the old literature looking for things, such as discussions of stored program computers. This was not the question Turing was addressing in 1936. The machine in the paper was not being offered as a new technique for real computers to take advantage of, nor even as a computer architecture research topic.

- Had Turing's attention been focused on that question afterward, giving him an opportunity to place it in its proper context, he would have answered it fairly. We know this, because he did. In 1950 he discusses the Analytical Engine at some length, mentions that Babbage held the Lucasian chair from 1828 to 1839, and calls the engine a universal digital computerAlan M. Turing, "Computing Machinery and Intelligence," Mind 59, no. 236 (October 1950): 433–460, the Analytical Engine at 439.. The acknowledgment was generous. Universality is the very property later writers would wish to reserve for Turing, yet Turing assigns it to Babbage without hesitation and without a syllable suggesting a rival claim. + Had Turing’s attention been focused on that question afterward, giving him an opportunity to place it in its proper context, he would have answered it fairly. We know this, because he did. In 1950 he discusses the Analytical Engine at some length, mentions that Babbage held the Lucasian chair from 1828 to 1839, and calls the engine a universal digital computerAlan M. Turing, "Computing Machinery and Intelligence," Mind 59, no. 236 (October 1950): 433–460, the Analytical Engine at 439.. The acknowledgment was generous. Universality is the very property later writers would wish to reserve for Turing, yet Turing assigns it to Babbage without hesitation and without a syllable suggesting a rival claim.

- Which leaves the question the historian actually wants answered. Was Turing at a point on a continuum, arriving where several others were already arriving, or did he leap? Some of the continuum is visible. Emil Post published a nearly identical formulation in the same year, a worker moving between boxes in a symbol space, and Gandy's own study of the period sets out how many hands were reaching for the same thingEmil L. Post, "Finite Combinatory Processes—Formulation 1," The Journal of Symbolic Logic 1, no. 3 (September 1936): 103–105.. + Which leaves the question the historian actually wants answered. Was Turing at a point on a continuum, arriving where several others were already arriving, or did he leap? Some of the continuum is visible. Emil Post published a nearly identical formulation in the same year, a worker moving between boxes in a symbol space, and Gandy’s own study of the period sets out how many hands were reaching for the same thingEmil L. Post, "Finite Combinatory Processes—Formulation 1," The Journal of Symbolic Logic 1, no. 3 (September 1936): 103–105..

- Whatever the leap was, it was not a matter of machine sophistication. Of the two machines, Babbage's was the concrete one. He drew its parts and worked out its gearing, and devised a Mechanical Notation to keep track of what the assembly was doingCharles Babbage, "On a Method of Expressing by Signs the Action of Machinery," Philosophical Transactions of the Royal Society of London 116 (1826)., which was an abstraction of a sort, though not of Turing's sort. Turing's machine was never meant to be built. It was described in order to be reasoned about, and described plainly enough that everything it could ever do could be enumerated. The move was not toward a more capable machine but away from one. + Whatever the leap was, it was not a matter of machine sophistication. Of the two machines, Babbage’s was the concrete one. He drew its parts and worked out its gearing, and devised a Mechanical Notation to keep track of what the assembly was doingCharles Babbage, "On a Method of Expressing by Signs the Action of Machinery," Philosophical Transactions of the Royal Society of London 116 (1826)., which was an abstraction of a sort, though not of Turing’s sort. Turing’s machine was never meant to be built. It was described in order to be reasoned about, and described plainly enough that everything it could ever do could be enumerated. The move was not toward a more capable machine but away from one.

@@ -579,27 +713,27 @@

- The Turing Machine itself fails the first condition. It cannot be built, so the machine that started the Naturalism thesis cannot carry the thesis. Some other machine has to, and satisfying the third condition is what makes the substitution of the new machine valid. A realizable machine that is not a computation theoretic inconsequential variation of the Turing Machine is merely some other machine, and a catalog of observations of it is a catalog of nothing in particular. The obvious candidate is the computer already sitting on the desk, which descends from the concrete machine Babbage drew rather than from the paper one Turing described. It is realizable by construction, it is the machine people actually own and run, and were it to satisfy the third condition the thesis would be finished here, with no proposal to make and no book to write. It does not satisfy it. Establishing that, and locating precisely where it fails, is in part the business of this book. A machine that does satisfy all three conditions is the RT Machine of chapter . + The Turing Machine itself fails the first condition. It cannot be built, so the machine that started the Naturalism thesis cannot carry the thesis. Some other machine has to, and satisfying the third condition is what makes the substitution of the new machine valid. A realizable machine that is not a computation theoretic inconsequential variation of the Turing Machine is merely some other machine, and a catalog of observations of it is a catalog of nothing in particular. The obvious candidate is the computer already sitting on the desk, which descends from the concrete machine Babbage drew rather than from the paper one Turing described. It is realizable by construction, it is the machine people actually own and run, and were it to satisfy the third condition the thesis would be finished here, with no proposal to make and no book to write. It does not satisfy it. Establishing that, and locating precisely where it fails, is in part the business of this book. A machine that does satisfy all three conditions is the Realizable Machine of chapter . The capitals mark a term carrying a formal definition that displaces its ordinary English sense, as they do for Real. A Realizable Machine is not merely a machine that happens to be realizable, which the computer on the desk also is; it is a machine meeting all three of the conditions set out above.

- A computer architecture, like a Turing Machine, is an abstraction, and the box sitting on a person's desk is a realization of one. Being Turing Complete is a weaker requirement than being a computation theoretic inconsequential variation. Being Turing Complete is the more conventional metric used. An architecture is Turing Complete if, when running a program that a Turing Machine could run to the point of halting, a realization of it can fault only because a) the program logic told it to, b) the program has a flaw, or c) there is a mathematical fact standing in the way of execution. A shortage of a physical resource is not a fourth reason, provided the architecture can pause a program until a 'more memory' request is fulfilled. Otherwise the computer also runs the program to the point of halting. However, if the architecture itself stipulates a bound that a program can reach, such as a fixed address width or a fixed Integer width, then every realization of it must fail on some program that a Turing Machine would carry to completion. Consequently, no current conventional architecture of this era meets even this weaker requirement. + A computer architecture, like a Turing Machine, is an abstraction, and the box sitting on a person’s desk is a realization of one. Being Turing Complete is a weaker requirement than being a computation theoretic inconsequential variation. Being Turing Complete is the more conventional metric used. An architecture is Turing Complete if, when running a program that a Turing Machine could run to the point of halting, a realization of it can fault only because a) the program logic told it to, b) the program has a flaw, or c) there is a mathematical fact standing in the way of execution. A shortage of a physical resource is not a fourth reason, provided the architecture can pause a program until a ‘more memory’ request is fulfilled. Otherwise the computer also runs the program to the point of halting. However, if the architecture itself stipulates a bound that a program can reach, such as a fixed address width or a fixed Integer width, then every realization of it must fail on some program that a Turing Machine would carry to completion. Consequently, no current conventional architecture of this era meets even this weaker requirement.

- As for the Turing Machine, the infinite tape is not as large a hurdle as it might seem at first. For computational problems, the Turing Machine halts in a finite number of steps. Because the Turing Machine is limited to stepping the read/write head over one cell per machine execution step, only a finite amount of tape is ever used. But for a given computation, how much tape is that? Resolving this by assuming more tape is simply attached when needed is analogous to cheating in a 'guess the bigger number' game by declaring, "My number is always one bigger than the given number, so I will tell you my guess after you state your number." Some mathematicians suggest that what is meant by infinity is precisely a rule of this sort. For engineers building physical hardware, however, to state that a resource starts finite and expands incrementally over time is a very different proposition from being asked to install infinite memory on a machine in the first place. + As for the Turing Machine, the infinite tape is not as large a hurdle as it might seem at first. For computational problems, the Turing Machine halts in a finite number of steps. Because the Turing Machine is limited to stepping the read/write head over one cell per machine execution step, only a finite amount of tape is ever used. But for a given computation, how much tape is that? Resolving this by assuming more tape is simply attached when needed is analogous to cheating in a ‘guess the bigger number’ game by declaring, "My number is always one bigger than the given number, so I will tell you my guess after you state your number." Some mathematicians suggest that what is meant by infinity is precisely a rule of this sort. For engineers building physical hardware, however, to state that a resource starts finite and expands incrementally over time is a very different proposition from being asked to install infinite memory on a machine in the first place.

- In 1967, Marvin Minsky addressed this very topic, saying: "We need not think of the machine's tape as infinite. We imagine instead that the machine begins with a finite tape, but that, whenever an end is encountered, another unit of tape is attached." Marvin L. Minsky, Computation: Finite and Infinite Machines (Englewood Cliffs: Prentice-Hall, 1967), 167. In 1967, this was a perfectly natural thing to suggest, as computers utilized magnetic tape memory on manually mounted reels, and it was entirely possible for a computation to stop and request a new reel of tape to be mounted. Contemporary computer architectures do, in fact, achieve a similar effect through virtual memory. When physical RAM is depleted, the operating system pauses the active process and autonomously provisions apparent capacity by swapping memory pages out to auxiliary storage. So the graceful expansion Minsky described is already in place, and it works, right up until a stipulated bound is reached when it doesn't work any longer. Once the available swap space is exhausted or the address space is saturated, the operating system abruptly terminates the process. + In 1967, Marvin Minsky addressed this very topic, saying: "We need not think of the machine’s tape as infinite. We imagine instead that the machine begins with a finite tape, but that, whenever an end is encountered, another unit of tape is attached." Marvin L. Minsky, Computation: Finite and Infinite Machines (Englewood Cliffs: Prentice-Hall, 1967), 167. In 1967, this was a perfectly natural thing to suggest, as computers utilized magnetic tape memory on manually mounted reels, and it was entirely possible for a computation to stop and request a new reel of tape to be mounted. Contemporary computer architectures do, in fact, achieve a similar effect through virtual memory. When physical RAM is depleted, the operating system pauses the active process and autonomously provisions apparent capacity by swapping memory pages out to auxiliary storage. So the graceful expansion Minsky described is already in place, and it works, right up until a stipulated bound is reached when it doesn’t work any longer. Once the available swap space is exhausted or the address space is saturated, the operating system abruptly terminates the process.

- Notice where the failure actually sits. Expanding the memory available to a process is handled, and handled well. What is missing is the ability to continue rather than to terminate, and supplying that is software's work, much as Python supplies Integers of arbitrary range over a machine whose words are of fixed width. The difference is one of motive. Python's arbitrary range Integers are a convenience offered to the programmer, whereas here the software would be compensating for a bound the architecture imposed. This is the shape of the whole problem in miniature: an architecture is made to look complete by software written to work around it, rather than being complete by virtue of its own instructions. + Notice where the failure actually sits. Expanding the memory available to a process is handled, and handled well. What is missing is the ability to continue rather than to terminate, and supplying that is software’s work, much as Python supplies Integers of arbitrary range over a machine whose words are of fixed width. The difference is one of motive. Python’s arbitrary range Integers are a convenience offered to the programmer, whereas here the software would be compensating for a bound the architecture imposed. This is the shape of the whole problem in miniature: an architecture is made to look complete by software written to work around it, rather than being complete by virtue of its own instructions.

- The problem with the controller is more nuanced. The head of a Turing Machine must read and react at every step, and the only place a controller has to hold what it has seen is a branch in its own control path. So the Turing Machine uses its controller as memory. This is treated in detail in section , where a machine that reverses a string is analyzed and the consequence is proven: the number of states and arcs required grows exponentially with the width of a machine word, and for a word of any practical size the controller cannot be built at all. A machine room operator can mount another reel of tape. Nobody can mount a larger controller, because the controller is fixed at the start. The tape's limit can be handed outside the machine and dealt with there, as we have just seen. The controller's limit is sealed inside the model, where nothing can reach it. + The problem with the controller is more nuanced. The head of a Turing Machine must read and react at every step, and the only place a controller has to hold what it has seen is a branch in its own control path. So the Turing Machine uses its controller as memory. This is treated in detail in section , where a machine that reverses a string is analyzed and the consequence is proven: the number of states and arcs required grows exponentially with the width of a machine word, and for a word of any practical size the controller cannot be built at all. A machine room operator can mount another reel of tape. Nobody can mount a larger controller, because the controller is fixed at the start. The tape’s limit can be handed outside the machine and dealt with there, as we have just seen. The controller’s limit is sealed inside the model, where nothing can reach it.

@@ -607,7 +741,7 @@

- Any solution to the Entscheidungsproblem required, as a prerequisite, that Hilbert and Ackermann's effective procedures be formalized, since nothing can be proved about what has not been articulated. That prerequisite is a salient subset of the second condition, which asks for any statement in mathematics. Alonzo Church got there first. He offered a solution to the Entscheidungsproblem in terms of his lambda calculus, and with it a general account of what an effective procedure is, identifying effective calculability with λ-definability. Gödel found the identification unsatisfactory, and Church restated it in terms of Herbrand-Gödel general recursiveness, which is the form that reached printAlonzo Church, "An Unsolvable Problem of Elementary Number Theory," American Journal of Mathematics 58, no. 2 (April 1936): 345–363, the identification at 356. A preliminary statement was presented to the American Mathematical Society on 1935-04-19. The name 'Church's thesis' is due to Kleene. Church first framed the identification in terms of his own λ-definability. Gödel told him that this was thoroughly unsatisfactory, and Church restated it in terms of Herbrand-Gödel general recursiveness, which is the form that reached print. The exchange survives in a letter from Church to Kleene of 1935-11-29, quoted in Stephen C. Kleene, "Origins of Recursive Function Theory," Annals of the History of Computing 3, no. 1 (1981): 52–67, at 59.. Turing arrived at the same prerequisite by another route, identifying the effective procedure with a Turing Machine program, and argued for it from what a person does when he computes with pencil and paper. Neither claim is the sort of thing that admits of proof, since one side of each identity is an informal notion and offers nothing to prove against. What can be done is to argue for it, and Turing's argument was the more intuitive, which is what settled the controversyChurch himself conceded that the 'Turing machine' explanation had the advantage of "making the identification with effectiveness in the ordinary (not explicitly defined) sense evident immediately", without preliminary theorems. Alonzo Church, review of A. M. Turing, "On Computable Numbers, with an Application to the Entscheidungsproblem," The Journal of Symbolic Logic 2, no. 1 (March 1937): 42–43. It runs two pages, and is the clearest short statement of the difference, made by the man who stood to lose by it. Gödel, who had rejected Church's proposal, came to the same view, remarking to Hao Wang that the sharp concept of a mechanical procedure was not perceived before Turing, who brought us to the right perspective; Hao Wang, From Mathematics to Philosophy (London: Routledge and Kegan Paul, 1974), 85. How far Gödel ever accepted the thesis in Church's own form is disputed; see Martin Davis, "Why Gödel Didn't Have Church's Thesis," Information and Control 54 (1982): 3–24.. + Any solution to the Entscheidungsproblem required, as a prerequisite, that Hilbert and Ackermann’s effective procedures be formalized, since nothing can be proved about what has not been articulated. That prerequisite is a salient subset of the second condition, which asks for any statement in mathematics. Alonzo Church got there first. He offered a solution to the Entscheidungsproblem in terms of his lambda calculus, and with it a general account of what an effective procedure is, identifying effective calculability with λ-definability. Gödel found the identification unsatisfactory, and Church restated it in terms of Herbrand-Gödel general recursiveness, which is the form that reached printAlonzo Church, "An Unsolvable Problem of Elementary Number Theory," American Journal of Mathematics 58, no. 2 (April 1936): 345–363, the identification at 356. A preliminary statement was presented to the American Mathematical Society on 1935-04-19. The name ‘Church’s thesis’ is due to Kleene. Church first framed the identification in terms of his own λ-definability. Gödel told him that this was thoroughly unsatisfactory, and Church restated it in terms of Herbrand-Gödel general recursiveness, which is the form that reached print. The exchange survives in a letter from Church to Kleene of 1935-11-29, quoted in Stephen C. Kleene, "Origins of Recursive Function Theory," Annals of the History of Computing 3, no. 1 (1981): 52–67, at 59.. Turing arrived at the same prerequisite by another route, identifying the effective procedure with a Turing Machine program, and argued for it from what a person does when he computes with pencil and paper. Neither claim is the sort of thing that admits of proof, since one side of each identity is an informal notion and offers nothing to prove against. What can be done is to argue for it, and Turing’s argument was the more intuitive, which is what settled the controversyChurch himself conceded that the ‘Turing machine’ explanation had the advantage of "making the identification with effectiveness in the ordinary (not explicitly defined) sense evident immediately", without preliminary theorems. Alonzo Church, review of A. M. Turing, "On Computable Numbers, with an Application to the Entscheidungsproblem," The Journal of Symbolic Logic 2, no. 1 (March 1937): 42–43. It runs two pages, and is the clearest short statement of the difference, made by the man who stood to lose by it. Gödel, who had rejected Church’s proposal, came to the same view, remarking to Hao Wang that the sharp concept of a mechanical procedure was not perceived before Turing, who brought us to the right perspective; Hao Wang, From Mathematics to Philosophy (London: Routledge and Kegan Paul, 1974), 85. How far Gödel ever accepted the thesis in Church’s own form is disputed; see Martin Davis, "Why Gödel Didn’t Have Church’s Thesis," Information and Control 54 (1982): 3–24..

@@ -619,7 +753,7 @@

- The remainder is delivered by construction rather than by thesis, and it occupies much of this book. A tape cell is defined as a location in physical memory in section , and a symbol in computational terms in section . Logic follows from relay switch logic, as Shannon and others have already established. On top of logic sits the Peano Machine, a counter, which then serves as the definition of Peano Numbers. Where Gödel reduced logic to Peano Numbers, we run the other way and expand logic out of them. An axiomatic proof becomes a decider assembled from subroutine calls to the axioms. Frege's set theory becomes the analysis of a logic program against an enumeration of inputs. Russell's paradox becomes a machine that can be analyzed in the second order though it will never halt in the first, which requires the orders of analysis set out in section . The whole is drawn together in chapter , where the claim is that every statement a mathematician has ever made can be restated in this language. + The remainder is delivered by construction rather than by thesis, and it occupies much of this book. A tape cell is defined as a location in physical memory in section , and a symbol in computational terms in section . Logic follows from relay switch logic, as Shannon and others have already established. On top of logic sits the Peano Machine, a counter, which then serves as the definition of Peano Numbers. Where Gödel reduced logic to Peano Numbers, we run the other way and expand logic out of them. An axiomatic proof becomes a decider assembled from subroutine calls to the axioms. Frege’s set theory becomes the analysis of a logic program against an enumeration of inputs. Russell’s paradox becomes a machine that can be analyzed in the second order though it will never halt in the first, which requires the orders of analysis set out in section . The whole is drawn together in chapter , where the claim is that every statement a mathematician has ever made can be restated in this language.

@@ -627,7 +761,7 @@

- There is a remedy, and it occupies the chapters that follow. We will transform the Turing Machine into a modern architecture in a stepwise fashion, while ensuring that at each step the modifications are inconsequential to computation theoretic existence proofs and complexity class results. The transformation does not run in one direction only. On the Turing Machine side, the controller has to stop being used as memory. On the modern side, the fixed widths an architecture stipulates, those of an address and of an Integer, are what have to give way. The two meet in the middle, and the machine we arrive at is less strange than that might suggest. It separates the control path from the data path; it holds an instruction table; and it has a small register file. It looks modern. The differences from what we currently build are real but few, and the point of the exercise is that we could build it. That machine, the RT Machine of chapter , satisfies the first condition by being buildable and the third by the manner of its construction, and it inherits the second from the Turing Machine it was transformed out of. Neither the Turing Machine nor the computer on the desk is a Natural manifestation of mathematics. The RT Machine, standing between them, is. + There is a remedy, and it occupies the chapters that follow. We will transform the Turing Machine into a modern architecture in a stepwise fashion, while ensuring that at each step the modifications are inconsequential to computation theoretic existence proofs and complexity class results. The transformation does not run in one direction only. On the Turing Machine side, the controller has to stop being used as memory. On the modern side, the fixed widths an architecture stipulates, those of an address and of an Integer, are what have to give way. The two meet in the middle, and the machine we arrive at is less strange than that might suggest. It separates the control path from the data path; it holds an instruction table; and it has a small register file. It looks modern. The differences from what we currently build are real but few, and the point of the exercise is that we could build it. That machine, the Realizable Machine of chapter , satisfies the first condition by being buildable and the third by the manner of its construction, and it inherits the second from the Turing Machine it was transformed out of. Neither the Turing Machine nor the computer on the desk is a Natural manifestation of mathematics. The Realizable Machine, standing between them, is.

@@ -669,7 +803,7 @@

- It is not a requirement of a computer organization, nor of an architecture, that it be capable of physical realization. The abstract Turing Machine organization developed in chapter serves as an example. Instead, an abstract organization can serve other purposes, in this case as a stepping stone to another organization that can be realized. + It is not a requirement of a computer organization, nor of an architecture, that it be capable of physical realization. The abstract Turing Machine organization developed in chapter serves as an example. Instead, an abstract organization can serve other purposes, in this case as a stepping stone to another organization that can be realized.

@@ -698,7 +832,7 @@

- The common understanding of the word 'architecture' is what Hamacher, Vranesic, and Zaky call an 'organization'. For example, even the most experienced of architects will say things like a microprocessor has a "superscalar architecture", though whether a processor is scalar or superscalar is clearly a question of computer organization. + The common understanding of the word ‘architecture’ is what Hamacher, Vranesic, and Zaky call an ‘organization’. For example, even the most experienced of architects will say things like a microprocessor has a "superscalar architecture", though whether a processor is scalar or superscalar is clearly a question of computer organization.

@@ -774,7 +908,7 @@

- Here is an interesting question. Suppose that instead of running a machine, which is to say performing a first-order analysis, to discover what the machine outputs on a tape, a second-order analysis is used to determine this. Say for example the program is that of a decider, and it will print 'Y' or 'N'. It might be that for a given input the program wastes a lot of time going through the motions when the output is would be immediately apparent to an analyzer. Generally, such a second-order analyzer that determines the output of another program must interpret the instructions found in the analyzed program in order to find the output. Interpretation is almost always slower than directly running the machine to get its output, a first-order analysis, as a lot of additional work must be done. An analyzer looking for shortcuts would be merely discovering things that would have been missed optimization opportunities when the analyzed program was written. + Here is an interesting question. Suppose that instead of running a machine, which is to say performing a first-order analysis, to discover what the machine outputs on a tape, a second-order analysis is used to determine this. Say for example the program is that of a decider, and it will print ‘Y’ or ‘N’. It might be that for a given input the program wastes a lot of time going through the motions when the output is would be immediately apparent to an analyzer. Generally, such a second-order analyzer that determines the output of another program must interpret the instructions found in the analyzed program in order to find the output. Interpretation is almost always slower than directly running the machine to get its output, a first-order analysis, as a lot of additional work must be done. An analyzer looking for shortcuts would be merely discovering things that would have been missed optimization opportunities when the analyzed program was written.

@@ -790,7 +924,7 @@

- Russell's set formulation of section is worth taking up here. It is perfectly legal to write a Turing Machine that accepts or rejects a proposed element by evaluating a logical predicate on it, and R = \{x \mid x ∉ x\} is such a machine. Asked whether R is a member of R, it will not halt. First-order analysis therefore returns nothing about it at all, which is the condition described above under which the only remaining option is a higher-order analysis. So the machine, its input, and the whole system around it are written to a tape and reasoned about instead, which is what was done in that section and what we are doing here. R is not dismissed. It is promoted. + Russell’s set formulation of section is worth taking up here. It is perfectly legal to write a Turing Machine that accepts or rejects a proposed element by evaluating a logical predicate on it, and R = \{x \mid x ∉ x\} is such a machine. Asked whether R is a member of R, it will not halt. First-order analysis therefore returns nothing about it at all, which is the condition described above under which the only remaining option is a higher-order analysis. So the machine, its input, and the whole system around it are written to a tape and reasoned about instead, which is what was done in that section and what we are doing here. R is not dismissed. It is promoted.

@@ -806,7 +940,7 @@

- Orders of analysis are not formally constructed from fundamentals. A second-order analysis might be taken up to address a hole that directly running a machine can not fill. However, that doesn't say what order of machine it is that has the hole. That information might be immaterial. Hence, discussions of orders of analysis are bubbles of meaning, rather than constructed pyramids of meaning. + Orders of analysis are not formally constructed from fundamentals. A second-order analysis might be taken up to address a hole that directly running a machine can not fill. However, that doesn’t say what order of machine it is that has the hole. That information might be immaterial. Hence, discussions of orders of analysis are bubbles of meaning, rather than constructed pyramids of meaning.

@@ -859,7 +993,7 @@

- We can then assign a property to transform T called its doesn't change results property, as follows. If and only if: + We can then assign a property to transform T called its doesn’t change results property, as follows. If and only if:

@@ -867,11 +1001,11 @@ \forall j \colon r_{i,j} = r_{i.1,j} - eq: The doesn't change results condition for one machine + eq: The doesn’t change results condition for one machine

- then T doesn't change m_i results. Here we note that we are evaluating a specific machine m_i, so we must add the qualifier 'm_i results'. It might be that for another machine with another corresponding set of interesting inputs, the transform would lead to a new machine that produces different results. + then T doesn’t change m_i results. Here we note that we are evaluating a specific machine m_i, so we must add the qualifier ‘m_i results’. It might be that for another machine with another corresponding set of interesting inputs, the transform would lead to a new machine that produces different results.

@@ -966,7 +1100,7 @@ The conventional Turing Machine -

This definition comes from Hopcroft and Ullman's book with minor terminology changes to make it flow into the text here John E. Hopcroft and Jeffrey D. Ullman, Introduction to Automata Theory, Languages, and Computation (Reading: Addison-Wesley, 1979).. +

This definition comes from Hopcroft and Ullman’s book with minor terminology changes to make it flow into the text here John E. Hopcroft and Jeffrey D. Ullman, Introduction to Automata Theory, Languages, and Computation (Reading: Addison-Wesley, 1979)..

@@ -1205,11 +1339,11 @@ eq: Arcs required by the conventional reverse machine -

These equations show that the state controller size explodes with word width. It would be impractical to implement for all but the smallest of word sizes. This is one of the reasons that computation theory books use modest-sized symbol alphabets in their examples, perhaps the first few letters of the Latin alphabet, or the letter 's' for unary arithmetic. Previous sections discussed challenges transitioning the Turing Machine to a modern architecture due to the tape length, and discussed how this could be mitigated. In contrast, there is no practical mediation for implementing a Turing Machine controller even for modest-sized real problems.

+

These equations show that the state controller size explodes with word width. It would be impractical to implement for all but the smallest of word sizes. This is one of the reasons that computation theory books use modest-sized symbol alphabets in their examples, perhaps the first few letters of the Latin alphabet, or the letter ‘s’ for unary arithmetic. Previous sections discussed challenges transitioning the Turing Machine to a modern architecture due to the tape length, and discussed how this could be mitigated. In contrast, there is no practical mediation for implementing a Turing Machine controller even for modest-sized real problems.

-

This raises a question: if the Turing Machine is to instruct upon the limitations of real computation, what are the implications stemming from its state controller being impractical? As one such implication, when a Turing Machine proof shows that a number is computable, it doesn't necessarily instruct upon how it could be computed. When a reader picks up a text on applied number theory, also called computer arithmetic, he is unlikely to find a chapter on Turing Machines.

+

This raises a question: if the Turing Machine is to instruct upon the limitations of real computation, what are the implications stemming from its state controller being impractical? As one such implication, when a Turing Machine proof shows that a number is computable, it doesn’t necessarily instruct upon how it could be computed. When a reader picks up a text on applied number theory, also called computer arithmetic, he is unlikely to find a chapter on Turing Machines.

-

The observation runs the other way as well. There is not a single chapter dedicated to computation theory in Hennessy and Patterson's definitive textbook on computer architecture. John L. Hennessy and David A. Patterson, Computer Architecture: A Quantitative Approach, 6th ed. (Cambridge: Morgan Kaufmann, 2017).

+

The observation runs the other way as well. There is not a single chapter dedicated to computation theory in Hennessy and Patterson’s definitive textbook on computer architecture. John L. Hennessy and David A. Patterson, Computer Architecture: A Quantitative Approach, 6th ed. (Cambridge: Morgan Kaufmann, 2017).

@@ -1259,12 +1393,12 @@ The memory of emptiness -

Turing described a clerk writing symbols into squares, with the option to erase them. He states, "In some of the configurations in which the scanned square is blank (i.e. bears no symbol) the machine writes down a new symbol on the scanned square: in other configurations it erases the scanned symbol."Alan M. Turing, "On Computable Numbers, with an Application to the Entscheidungsproblem," Proceedings of the London Mathematical Society, Series 2, Volume 42 (1936): 231. So initially Turing's original article uses blank to mean no symbol. However, later Turing says, "We also give numbers to the symbols ... in particular, blank = S₀ ." ibid: 239..

+

Turing described a clerk writing symbols into squares, with the option to erase them. He states, "In some of the configurations in which the scanned square is blank (i.e. bears no symbol) the machine writes down a new symbol on the scanned square: in other configurations it erases the scanned symbol."Alan M. Turing, "On Computable Numbers, with an Application to the Entscheidungsproblem," Proceedings of the London Mathematical Society, Series 2, Volume 42 (1936): 231. So initially Turing’s original article uses blank to mean no symbol. However, later Turing says, "We also give numbers to the symbols ... in particular, blank = S₀ ." ibid: 239..

The abstract concept here is that of a container. A piece of paper is a container for symbols. When it contains no symbols, the container is empty. Only when we say the container is a piece of paper does it make sense to call it blank. But we are speaking of abstractions, not of paper. As a point of perspective, when computation theory texts are translated into French, the blank symbol is called the "symbole vide", or literally "empty symbol", as apparently blank paper is empty.

-

Because emptiness is a property of a container, Turing's first statement can be modeled with a sequence of sets. For a Turing Machine tape, each sequence member is either an empty set or a singleton set. In the language of mathematics an empty tape can be defined as a empty sets:

+

Because emptiness is a property of a container, Turing’s first statement can be modeled with a sequence of sets. For a Turing Machine tape, each sequence member is either an empty set or a singleton set. In the language of mathematics an empty tape can be defined as a empty sets:

@@ -1320,21 +1454,21 @@

If we view the empty symbol from within the definition of machine B, it is a category error. It is a sequence element rather than a container. Even if it were allowed that a position within a sequence were a container and thus could have the property of being empty, the symbol represents that there is no symbol in the very location it is found. Hence, to ascribe an appropriate meaning to the empty symbol, the entire system must be kept including the mapping to machine A, then the empty symbol found on machine B means that if machine A were used instead, the same location in the tape sequence would be an empty set. However, this feels unsatisfactory, because in all other respects, machine B is a perfectly well-defined Turing Machine all by itself.

-

Going back to Turing's moniker of blank does limit the focus solely to machine B, because the original paper by Turing states the definition of blank meaning "no symbol", as was already discussed. Calling it blank is merely the same name colored by the engineering of wood pulp. However, there is a pleasing property of the blank, i.e. the empty symbol, that is valuable and unique to it. It prints as a space in diagrams given in text books and in the output of Turing Machine simulators. There is an analogous symbolic system that also has this characteristic, and is being leveraged in these situations: the ASCII hex 20, called SP. It prints as a space leaving an area of the paper it is printed on blank. Perhaps a better name for the ersatz empty symbol is SP.

+

Going back to Turing’s moniker of blank does limit the focus solely to machine B, because the original paper by Turing states the definition of blank meaning "no symbol", as was already discussed. Calling it blank is merely the same name colored by the engineering of wood pulp. However, there is a pleasing property of the blank, i.e. the empty symbol, that is valuable and unique to it. It prints as a space in diagrams given in text books and in the output of Turing Machine simulators. There is an analogous symbolic system that also has this characteristic, and is being leveraged in these situations: the ASCII hex 20, called SP. It prints as a space leaving an area of the paper it is printed on blank. Perhaps a better name for the ersatz empty symbol is SP.

We therefore keep both words, and give each one job. Emptiness is a property, and a property belongs to a container: a cell of machine A either holds a symbol or it does not. SP is a symbol, and a symbol belongs to an alphabet: a cell of machine B always holds one, and SP is what it holds where a cell of machine A would have been empty. Machine B has no empty cells at all, only cells holding SP. Where this book speaks of emptiness it means the property, and where it writes SP it means the symbol standing in for it.

There is more to consider about the former blank, former empty, now SP symbol. Computational theorists often describe a two-step process. First, a Turing Machine writes a string to be analyzed onto a tape initially filled with all SP symbols; second, the tape is moved to another Turing Machine whose purpose is to recognize a property of the string being analyzed. In this process, a tape is used as a communication medium, so it follows that communication theory can be applied to this messaging scenario. Messages sent on a channel must be formatted so they can be placed on the channel and later recovered. The specification for such structuring is called a communication protocol. The protocol implies a higher authority, as both the sender and the receiver must have knowledge of the same designed-in protocol. In a sense, a protocol is the meta-message sent before the message itself.

-

The intention of the later innovation of placing the SP symbol in Γ but not in Σ appears to be that of making the SP symbol serve additional duty: firstly as a default value for a cell that has never been written to, secondly to effect erasure, and now thirdly as a control component of a communications protocol. Accordingly then, this protocol was designed by mathematicians with a specific goal. They intended to make it impossible for programmers to send 'to be analyzed' string messages that cannot always be recovered by a receiving recognizer Turing Machine. In short, the SP symbol also serves as an in-band EOM (end of message) symbol.

+

The intention of the later innovation of placing the SP symbol in Γ but not in Σ appears to be that of making the SP symbol serve additional duty: firstly as a default value for a cell that has never been written to, secondly to effect erasure, and now thirdly as a control component of a communications protocol. Accordingly then, this protocol was designed by mathematicians with a specific goal. They intended to make it impossible for programmers to send ‘to be analyzed’ string messages that cannot always be recovered by a receiving recognizer Turing Machine. In short, the SP symbol also serves as an in-band EOM (end of message) symbol.

Firstly note that excluding SP from Σ does not prevent a Turing Machine from writing SP to a tape to be used as a message. If a message writing machine wants to write an SP to the tape, it merely writes a different symbol then erases it.

Secondly, once it is realized that SP symbols can be written to tape as messages, the very reason of excluding it from Γ, to guarantee that received messages are parsable, is lost. A message authoring machine could, for example, happily output a variable length array of strings while using an arbitrary number of empty symbols to separate the elements. A receiving machine could then not know how many elements it should try to recover, or when to say a string of SP symbols is sufficiently long to not be considered a separator. Nothing in the Turing Machine definition prevents this.

-

Furthermore, the Universal Turing Machine can act as the aforementioned higher authority. In this case, Turing Machines, and sometimes their tapes, are written to a Universal Turing Machine's tape. Consequently, it is not only possible to want to terminate multiple strings in series, as mentioned in the prior paragraph; the termination issue itself is embedded in proofs such as the Halting Problem.

+

Furthermore, the Universal Turing Machine can act as the aforementioned higher authority. In this case, Turing Machines, and sometimes their tapes, are written to a Universal Turing Machine’s tape. Consequently, it is not only possible to want to terminate multiple strings in series, as mentioned in the prior paragraph; the termination issue itself is embedded in proofs such as the Halting Problem.

-

A common technique programmers use for embedding strings with terminators within other strings is to embed escape characters. This is another in-band protocol, and it is inefficient. The escape sequences grow exponentially with nesting levels. Also, such strings with embedded escape sequences cannot be interpreted without knowledge of the intended nesting level of the string author, which requires the interpreter of the string to know the author's intentions.

+

A common technique programmers use for embedding strings with terminators within other strings is to embed escape characters. This is another in-band protocol, and it is inefficient. The escape sequences grow exponentially with nesting levels. Also, such strings with embedded escape sequences cannot be interpreted without knowledge of the intended nesting level of the string author, which requires the interpreter of the string to know the author’s intentions.

For the modified computational Turing Machine presented later in this chapter, there will be a separate control alphabet of predefined symbols, as for the Hopcroft-Ullman interpretation. However, so as to support recursion, and because the current scheme does not achieve protocol guarantees, the predefined alphabet will be part of the alphabet Σ rather than distinct from it. Programmers then must explicitly design communication protocols that suit the problems they are working on, and those protocols become opaque for examination by a higher authority. Note that this does not preclude a programmer from using SP as an EOM marker, rather it removes the embedding of that protocol from the Turing Machine definition.

@@ -1343,12 +1477,12 @@ - - The computation theoretic RT Machine + + The computation theoretic Realizable Machine

This chapter presents a modified computation theoretic Turing Machine with three structural additions. First, it separates control flow from data flow, ensuring that payload symbols do not needlessly expand the control state machine. Second, it unifies the control and data symbol sets into a single alphabet to natively support explicit communications protocols even in the presence of recursion and self-recursion. Finally, it implements a cascading next-state evaluation hierarchy, permitting the programmer to formally define and handle meta-symbols while also making it more convenient to program communication protocols.

- + The unspecified meta-symbol

In the first edition of this book, I introduced a "read-only after write" rule while working towards an architectural Turing Machine because conventional computer architectures do not maintain a concept of empty memory. The approach described in this section integrates the "read-only after write" into a computation theoretic machine by designing in the concept of being unspecified, which then displaces the concept of a cell being empty.

@@ -1376,16 +1510,16 @@

An example of effectively unspecified data would be a program that reverses a string without looking at the values being reversed. A string reverse function need not inspect the value of the string; it only needs to recognize the structural boundaries established by the writing protocol. Yet the conventional Turing Machine is incapable of doing this, and worse, as we saw, there is an explosion in the number of states for the reverse string controller against word length.

-

Recall the suggestion earlier in this section that "perhaps an algorithm could be analyzed for this very quality of not ever making decisions based on unspecified data." In this capacity, the unspecified symbol functions as a test probe. Such an analysis can be done with a two-layer architecture: a first-order machine under study and a second-order machine performing the analysis. The unspecified symbol resides within the alphabet of the second-order machine, which possesses the authority not only to move the symbol but also to base logic upon it. However, demoting the unspecified marker from a meta-symbol to a standard decision symbol leaves the second-order machine without a meta-symbol of its own. In a strictly layered architecture, analogous to Russell and Whitehead's hierarchy of types, a programmer could define unspecified-0, unspecified-1, and so forth, explicitly embedding the order as a unique identifier. However, if the system lacks this strict stratification, the layering strategy collapses. This occurs when a statement operates as an independent island of meaning, analogous to Gödel's unprovable truths, or when an analyzer is tasked with evaluating itself, as in Turing's halting proof. Therefore, whether an unspecified meta-symbol can be deployed successfully depends entirely upon the structural boundaries of the specific system under test.

+

Recall the suggestion earlier in this section that "perhaps an algorithm could be analyzed for this very quality of not ever making decisions based on unspecified data." In this capacity, the unspecified symbol functions as a test probe. Such an analysis can be done with a two-layer architecture: a first-order machine under study and a second-order machine performing the analysis. The unspecified symbol resides within the alphabet of the second-order machine, which possesses the authority not only to move the symbol but also to base logic upon it. However, demoting the unspecified marker from a meta-symbol to a standard decision symbol leaves the second-order machine without a meta-symbol of its own. In a strictly layered architecture, analogous to Russell and Whitehead’s hierarchy of types, a programmer could define unspecified-0, unspecified-1, and so forth, explicitly embedding the order as a unique identifier. However, if the system lacks this strict stratification, the layering strategy collapses. This occurs when a statement operates as an independent island of meaning, analogous to Gödel’s unprovable truths, or when an analyzer is tasked with evaluating itself, as in Turing’s halting proof. Therefore, whether an unspecified meta-symbol can be deployed successfully depends entirely upon the structural boundaries of the specific system under test.

 - + The modifications

The specific architectural modifications are as follows:

- - + +
  1. There is one unified alphabet Σ to which both status symbols and data symbols belong.
  2. The machine utilizes a Moore-style programmed state controller, with zero or one instruction specified per state, so that instructions are independent and managed separately from state transitions.
    3. @@ -1403,7 +1537,7 @@
- List . The architectural modifications that distinguish the RT Machine + List . The architectural modifications that distinguish the Realizable Machine

Here the subscript i is a device used to emphasize that q_i and q_{i+1} can be any members of the total set of states, Q. State q_0 refers specifically to the initial state. Also note, later the spartan q will be used to denote the contents of the q register, the current state register.

@@ -1433,36 +1567,36 @@ - - The RT Machine fixed part + + The Realizable Machine fixed part - - + + \mathit{MF} = (\mathit{QF}, \mathit{ΣF}, \mathit{AF}) - eq: MF, the fixed part of the RT Machine + eq: MF, the fixed part of the Realizable Machine

In the following, the middle dot acts as a namespace operator, N{·}x. By doing this we assure there will be no aliasing with the symbols provided by the programmer when he defines a programmed state controller.

The set of predefined states:

- - + + \mathit{QF} = \{\mathit{QF}{·}\mathtt{initial}\} - eq: QF, the set of predefined states + eq: QF, the set of predefined states -

The state controller always starts in the QF·initial state. This is a symbol representing the state; it is not a register that holds a state. The 'F' values are all fixed.

+

The state controller always starts in the QF·initial state. This is a symbol representing the state; it is not a register that holds a state. The ‘F’ values are all fixed.

The programmer cannot add instructions to the machine definition, so there are no symbol aliasing issues here:

- - + + \begin{aligned} \mathit{AF} = \{& \\ @@ -1475,19 +1609,19 @@ \} \end{aligned} - eq: AF, the set of predefined instructions + eq: AF, the set of predefined instructions

where σ must be in Σ.

The set of predefined symbols:

- - + + \mathit{ΣF} = \{\mathit{ΣF}{·}\mathtt{leftmost}, \mathit{ΣF}{·}\mathtt{rightmost}\} - eq: ΣF, the set of predefined symbols + eq: ΣF, the set of predefined symbols

@@ -1497,15 +1631,15 @@ - + Machine variables - - + + \mathit{MV} = (q, s, d) - eq: MV, the machine variables + eq: MV, the machine variables

q: is the current state of the machine.

@@ -1517,46 +1651,46 @@ - + Programmable part - - + + \mathit{MP} = (\mathit{QP}, \mathit{ΣP}, \mathit{ΛP}, Δ_0, Δ_1, Δ_2, Δ_3, \mathit{HP}) - eq: MP, the programmable part of the RT Machine + eq: MP, the programmable part of the Realizable Machine

A set of programmed state symbols:

- - + + \mathit{QP} - eq: QP, the set of programmed state symbols + eq: QP, the set of programmed state symbols

A set of programmed data symbols:

- - + + \mathit{ΣP} - eq: ΣP, the set of programmed data symbols + eq: ΣP, the set of programmed data symbols

The programmed instructions. A set of pairs of the form:

- - + + \mathit{ΛP} = \{ \langle q_i, a \rangle, \dots \} - eq: ΛP, the programmed instructions, as state and action pairs + eq: ΛP, the programmed instructions, as state and action pairs

where q_i is matched to the current state, and a is a member of \mathit{A}. @@ -1564,74 +1698,74 @@

The conditional transition table. A set of state transition triples; each triple is of the form:

- - + + Δ_0 = \{ \langle q_i, σ, q_{i+1} \rangle, \dots \} - eq: Δ₀, the conditional transition table + eq: Δ₀, the conditional transition table

Here q_i and q_{i+1} are two states from the total set of Q. They need not be distinct. While the machine is running, state q_i is to be matched against the contents of the q register, the current state. Symbol σ is a member of the total set Σ and is to be matched against the contents of the s register, the machine status. When q_i matches the current state and σ matches the current status, then q_{i+1} becomes the next-state.

The state default transition table. A set of state transition pairs; each pair is of the form:

- - + + Δ_1 = \{ \langle q_i, q_{i+1} \rangle, \dots \} - eq: Δ₁, the state default transition table + eq: Δ₁, the state default transition table

where q_i is matched to the current state, and upon a match q_{i+1} will be taken as the next-state.

The status default transition table. A set of state transition pairs; each pair is of the form:

- - + + Δ_2 = \{ \langle σ, q_{i+1} \rangle, \dots \} - eq: Δ₂, the status default transition table + eq: Δ₂, the status default transition table

where σ matches the symbol in s register, and upon a match q_{i+1} will be taken as the next-state.

The global default next-state:

- - + + Δ_3 = q_{i+1} - eq: Δ₃, the global default next-state + eq: Δ₃, the global default next-state

This is the transition of last resort. It is unconditional; the next-state becomes q_{i+1}.

A set of programmer-defined halting states:

- - + + \mathit{HP} - eq: HP, the set of programmed halting states + eq: HP, the set of programmed halting states - - The RT Machine definition in total + + The Realizable Machine definition in total - - + + M = (q, s, d, Q, Σ, A, Λ, Δ, H) - eq: The RT Machine in total + eq: The Realizable Machine in total @@ -1639,77 +1773,77 @@

The complete set of states, uniting the fixed predefined states and the programmed states:

- - + + Q = \mathit{QF} \cup \mathit{QP} - eq: Q, the complete state set, fixed united with programmed + eq: Q, the complete state set, fixed united with programmed

The complete set of symbols, uniting the fixed control symbols and the programmed data symbols:

- - + + Σ = \mathit{ΣF} \cup \mathit{ΣP} - eq: Σ, the complete symbol set, fixed united with programmed + eq: Σ, the complete symbol set, fixed united with programmed

All members of the set of available instructions are fixed:

- - + + A = \mathit{AF} - eq: A, the instruction set, entirely fixed + eq: A, the instruction set, entirely fixed

The table of state-instruction pairs is strictly programmed.

- - + + Λ = \mathit{ΛP} - eq: Λ, the instruction table, entirely programmed + eq: Λ, the instruction table, entirely programmed

The ordered sequence of next-state transition rules:

- - + + Δ = [Δ_0 \mid Δ_1 \mid Δ_2 \mid Δ_3] - eq: Δ, the transition rules as an ordered sequence + eq: Δ, the transition rules as an ordered sequence

The set of halt states is strictly programmed, and thus could be empty.

- - + + H = \mathit{HP} - eq: H, the halt state set, entirely programmed + eq: H, the halt state set, entirely programmed - - Computation theoretic RT Machine executor + + Computation theoretic Realizable Machine executor

Computer was at one time a job title. Turing described a clerk following directions to cause his machine to go, thus implying that mathematicians are mere automata chained to following the procedures they derive. So then an executor can be a person, perhaps a student who is studying the computation theoretic machine, and has a homework assignment of showing what it does.

-

The executor takes the RT Machine through three stages of execution: initialization, programmed control, and halting. While going through these stages, the executor gives the \mathit{MV} variables values.

+

The executor takes the Realizable Machine through three stages of execution: initialization, programmed control, and halting. While going through these stages, the executor gives the \mathit{MV} variables values.

This description assumes support is programmed in for the first-order unspecified symbol.

- + Initialization stage

Before the first cycle begins, a tape is selected and mounted. The read/write head is positioned over the leftmost tape cell. The machine variables are initialized as follows:

@@ -1724,10 +1858,10 @@ - + Programmed control stage - + Phase 1: Instruction issue and execution @@ -1741,14 +1875,14 @@ - + Phase 2: The state transition

Following the completion of the instruction, the programmed controller evaluates the next-state by cascading through the ordered sequence of transition rules Δ = [Δ_0 \mid Δ_1 \mid Δ_2 \mid Δ_3]. The evaluation follows this hierarchy, stopping and branching at the first valid match. Upon not finding a next-state transition, the machine hangs.

 - + Halting stage

If, after the state transition phase completes, the current state is a member of H, the machine halts. Otherwise, the execution continues from the programmed control stage.

@@ -1756,15 +1890,15 @@ - - The RT Machine string reverse + + The Realizable Machine string reverse -

Because the RT Machine separates the data path from the control path, it is possible to reverse a string without inspecting the payload. The programmed controller only needs to recognize the structural boundaries of the data protocol. When a payload symbol is encountered, the controller executes a read(d) instruction, placing the value into the data register, which is not examined for decision-making purposes. When a value is used to base a decision upon, the controller executes a read(s), placing the value into the status register. Because instructions are bound to states rather than transitions, reading and stepping are distinct states. This combination of features results in a controller that has threads of serialized execution.

+

Because the Realizable Machine separates the data path from the control path, it is possible to reverse a string without inspecting the payload. The programmed controller only needs to recognize the structural boundaries of the data protocol. When a payload symbol is encountered, the controller executes a read(d) instruction, placing the value into the data register, which is not examined for decision-making purposes. When a value is used to base a decision upon, the controller executes a read(s), placing the value into the status register. Because instructions are bound to states rather than transitions, reading and stepping are distinct states. This combination of features results in a controller that has threads of serialized execution.

- - + + - # RT Machine String Reverse + # Realizable Machine String Reverse # Domains: # ΣP = {0 ,1 ,EOM ,EOR ,SP} # Σ_data = ΣP \ {EOM ,EOR ,SP} @@ -1903,31 +2037,31 @@ δ: Q·Done - Code . The RT Machine string reverse controller + Code . The Realizable Machine string reverse controller - - + +
RTM reverse machine
- Figure . The RT Machine that reverses a binary string + Figure . The Realizable Machine that reverses a binary string - - Analysis of the RT reverse machine + + Analysis of the Realizable reverse machine

The number of states is constant at 24 independent of how much data is to be reversed. No arc refers to a payload value.

The form of this diagram shows a lead-in, a long loop, and a tail leading to done. This is suggestive of code followed by a while loop that breaks out, with further code completing the program.

- - + + - void RT·reverse_string() { + void Realizable·reverse_string() { // Initialization: Scan to EOM and setup the EOR boundary read('s'); while(s != EOM){right(); read('s');} @@ -1979,13 +2113,13 @@ return; } - Code . The RT reverse controller written as C + Code . The RT reverse controller written as C -

The total number of steps for reversing an n symbol string using the RT Machine:

+

The total number of steps for reversing an n symbol string using the Realizable Machine:

- - + + \text{steps} = \begin{cases} @@ -1993,13 +2127,13 @@ 4.5n^2 + 11.5n + 5 & \text{if } n \ge 1 \end{cases} - eq: Steps taken by the RT reverse machine + eq: Steps taken by the Realizable reverse machine - + Two-headed reverse string example

The reverse string machine spends a lot of time shuttling the head between two context areas: one for the original string, and one for the resulting reversed string. This suggests that a two-head version would be faster. The following is the two-head state machine definition:

@@ -2007,7 +2141,7 @@ - # RT Two-Head String Reverse + # Realizable Two-Head String Reverse # input: (σ ∈ Σ)* EOM (starting on the leftmost cell) # output: SP* EOM (σ ∈ Σ in reverse)* EOR @@ -2109,18 +2243,18 @@ Code . The two-headed string reverse controller - - + +
RTM 2 hd reverse machine
- Figure . A two-headed RT Machine that reverses a binary string + Figure . A two-headed Realizable Machine that reverses a binary string - + Analysis of the two-headed reverse string machine

The number of states has dropped from 24 to 18, while the speed increase is dramatic, with the former quadratic performance becoming linear performance. The total number of steps for reversing an n symbol string using a two-head RT architecture:

@@ -2135,7 +2269,7 @@ 9n + 4 & \text{if } n \ge 1 \end{cases} - eq: Steps taken by the two-headed RT reverse machine + eq: Steps taken by the two-headed Realizable reverse machine

The two paths through the state machine, and the one loop, translate well into code:

@@ -2143,7 +2277,7 @@ - void RT·reverse_string_2_head() { + void Realizable·reverse_string_2_head() { // Phase 1: Both heads scan right to the EOM pivot read('s' ,0); while(s != EOM){right(0); right(1); read('s' ,0);} @@ -2196,8 +2330,8 @@ - - The RT Machine design + + The Realizable Machine design @@ -2208,23 +2342,23 @@ Figure . A Turing Machine -

The prior chapter on the computation theoretic RT Machine, chapter , serves as the architectural template, with only a few modifications. The architecture requires explicit data rather than accepting meta-symbols like 'unspecified' as presumed initial values 'by definition'. Actual values are transacted. Now that data and control have been separated, the controller is practical to implement, and even more so because it was defined in terms of tables that can be built in hardware. As the read status instruction returns the cell type, it will in its current form be able to return 'rightmost', so the right end of the tape can be detected. In order to extend the tape, the machine will stop and ask the operator to mount a new reel. This could be signaled when the user attempts to step right of rightmost, either by a panel light that illuminates upon the machine finding a rightmost status, or by the program printing a message on the console teletype. As this is a constant-time operation, it is computation theoretic inconsequential.

+

The prior chapter on the computation theoretic Realizable Machine, chapter , serves as the architectural template, with only a few modifications. The architecture requires explicit data rather than accepting meta-symbols like ‘unspecified’ as presumed initial values ‘by definition’. Actual values are transacted. Now that data and control have been separated, the controller is practical to implement, and even more so because it was defined in terms of tables that can be built in hardware. As the read status instruction returns the cell type, it will in its current form be able to return ‘rightmost’, so the right end of the tape can be detected. In order to extend the tape, the machine will stop and ask the operator to mount a new reel. This could be signaled when the user attempts to step right of rightmost, either by a panel light that illuminates upon the machine finding a rightmost status, or by the program printing a message on the console teletype. As this is a constant-time operation, it is computation theoretic inconsequential.

Because a Turing Machine can only reach another cell further out on the tape by stepping to it, space complexity and time complexity are related. A program that runs for ten steps can consume at most ten cells of tape. However, if that program merely bounces between two cells, it will require less space, precisely two cells. As another example, a program that counts the number of characters on its input tape using Hindu-Arabic notation will execute in asymptotically linear time, as demonstrated in section , which analyses the increment operation. Its working footprint, however, will be logarithmic in space complexity, because that is how fast an Hindu-Arabic representation grows with a count.

-

If a program executed at the speed of a human operator, the operator would likely abandon the process before it finished. This highlights a necessary attribute of good software: utility. It also exposes a limitation of pure computation theory, which abstracts away physical time. Nevertheless, formal analysis remains a necessity. Consider an exponential-time program processing worst-case operands: its execution time explodes relative to input length, rapidly exceeding the age of the universe. In such extremes, empirical wall-clock measurement becomes superfluous. Computation theory classifies a program's behavior, which establishes structural implications for wall-clock time, rather than calculating absolute durations.

+

If a program executed at the speed of a human operator, the operator would likely abandon the process before it finished. This highlights a necessary attribute of good software: utility. It also exposes a limitation of pure computation theory, which abstracts away physical time. Nevertheless, formal analysis remains a necessity. Consider an exponential-time program processing worst-case operands: its execution time explodes relative to input length, rapidly exceeding the age of the universe. In such extremes, empirical wall-clock measurement becomes superfluous. Computation theory classifies a program’s behavior, which establishes structural implications for wall-clock time, rather than calculating absolute durations.

The same can be said for space complexity. Suppose a program doubled its memory footprint each time its input string increased by one. If an individual proton could hold one bit of memory, say via its spin, an input increase of merely 270 characters for such a program would exhaust all the protons in the universe. Allocating a cell of space requires the machine to take a step, so time complexity is at least equal to space complexity. If a computer ran at 10 GHz and a step required 10^{-10} seconds, this same extension would require 6 \times 10^{63} years. For perspective, the universe is approximately 1.4 \times 10^{10} years old.

This book provides the transformational steps needed to go from the Turing Machine to realized machines, and one objective is to recover some correspondence between the machine steps of the model and the wall-clock time the machine takes to run. Given this, the system operator changing tapes creates a step that is disproportionately longer than the other steps, a factor that requires architectural mediation, but will always remain. For example, this structural penalty resurfaces in the form of cache misses causing a machine to reach into system memory, or worse, page faults, requiring a machine to go back to disk.

-

As a possible practical solution, note that if the program does not exhaust the current tape, the operator will never be called. How much tape is required to ensure this? An analyst could choose worst-case operands and measure the footprint when the program runs. At first, this appears to be the familiar 'my number is bigger than your number, I'll tell you mine after you tell me yours' game, which can be viewed as the definition for the countable infinity. However, there is a loophole. Running the system once with the worst-case operands to establish time and space ceilings guarantees the program can be allocated sufficient resources later for other operands. This is a practical approach, provided the program is a workhorse utility rather than an algorithm searching for a solution to an unsolved problem, and thus only needs to be run once.

+

As a possible practical solution, note that if the program does not exhaust the current tape, the operator will never be called. How much tape is required to ensure this? An analyst could choose worst-case operands and measure the footprint when the program runs. At first, this appears to be the familiar ‘my number is bigger than your number, I’ll tell you mine after you tell me yours’ game, which can be viewed as the definition for the countable infinity. However, there is a loophole. Running the system once with the worst-case operands to establish time and space ceilings guarantees the program can be allocated sufficient resources later for other operands. This is a practical approach, provided the program is a workhorse utility rather than an algorithm searching for a solution to an unsolved problem, and thus only needs to be run once.

Another practical solution, one that also applies to the first run of the program, is to analyze the logic to compute the time per step and extrapolate the total execution time. This is structurally viable only for specific programs; for others, tracing a path through the execution logic is as computationally complex as running the program itself, taking us back to the analysis in the prior paragraph. However, this phenomenon restricts itself to specific programmatic classes. Consider the previously mentioned examples of the ten step machine, the machine bouncing between two cells, and the Hindu-Arabic counting machine. All of those programs can be, indeed were, analyzed before they were run.

Suppose a controller is not analyzed to determine its computation theoretic complexity, or even tested against worst-case inputs, but is instead run with random or everyday input to gather performance measurements. This process is called profiling. After many runs, a programmer might surmise the behavioral limits of the program. However, many programs are neither linear systems nor smooth functions. With a different set of inputs than those used for profiling, perhaps even values adjacent to prior inputs, the program behavior can shift drastically. Consider the Pentium divider. It did not matter how many millions of times the result was accurate; the fact remains that customers later found inputs that yielded wildly inaccurate results. Only by elevating the analysis to the structural logic of the code can such an eventuality from an erroneous program be categorically ruled out. This is why the K5 transcendental function development project included a proof writing phase; see Thomas Walker Lynch, A. Ahmed, M. Schulte, T. Callaway, and R. Tisdale, "The K5 Transcendental Functions," Proceedings of the 12th IEEE Symposium on Computer Arithmetic, 1995. DOI: 10.1109/ARITH.1995.465368.

- + Head Unit, HU

An HU contains a head and a local controller. The local controller supports these instructions:

@@ -2239,7 +2373,7 @@ List . The instructions supported by the head unit controller -

On this model of machine, the HU status is identical to the indicated cell's type. The cell type is not read from the tape; rather, it is derived from the head's physical relationship to the ends of the tape. With a physical tape drive, an unused leader and trailer are required to prevent the tape from departing from the reels, a condition difficult to reverse. Therefore, the format, or physical, markers will be on the ends of the usable portion of the tape, rather than on the physical end of the tape. Consequently, the HU works in conjunction with the TTU to derive the status. (The tape transport unit, the TTU, is discussed in the next section, section .) As established in section , cell types are:

+

On this model of machine, the HU status is identical to the indicated cell’s type. The cell type is not read from the tape; rather, it is derived from the head’s physical relationship to the ends of the tape. With a physical tape drive, an unused leader and trailer are required to prevent the tape from departing from the reels, a condition difficult to reverse. Therefore, the format, or physical, markers will be on the ends of the usable portion of the tape, rather than on the physical end of the tape. Consequently, the HU works in conjunction with the TTU to derive the status. (The tape transport unit, the TTU, is discussed in the next section, section .) As established in section , cell types are:

@@ -2256,11 +2390,11 @@

For a realized TTU, data can only be read or written when there is relative motion between the head and the tape. Consequently, in an HU implementation, data and status registers are updated from an internal buffer that acquires data during head motion. Conversely, a write value is buffered until subsequent head motion provides the opportunity to write it to the tape.

-

Because values can only be read or written to a tape when the tape is in motion, it is advantageous to exaggerate the motion of step instructions and to cache a small number of values. In addition, most architectures that make use of a tape drive will attempt to leverage high throughput in an effort to hide high latency by reading or writing blocks of symbols per tape access. Note, however, that the basic RT Machine organization lacks the core memory required for buffering blocks.

+

Because values can only be read or written to a tape when the tape is in motion, it is advantageous to exaggerate the motion of step instructions and to cache a small number of values. In addition, most architectures that make use of a tape drive will attempt to leverage high throughput in an effort to hide high latency by reading or writing blocks of symbols per tape access. Note, however, that the basic Realizable Machine organization lacks the core memory required for buffering blocks.

 - + Tape transport unit, TTU

The TTU is the component where tapes are mounted and unmounted, and where read/write head units are installed. When multiple HUs are plugged in, the controller passes control among them so it appears that heads never collide. The TTU controller has these components: @@ -2295,7 +2429,7 @@ - + The customer programmed control unit, CPCU

The controller is programmed via patch panels. The panels would look something like what is shown in the following ASCII art blocks. Note that ● indicates an illuminated indicator light, whereas ○ is not illuminated. [/] represents an open toggle switch, while [—] is a closed one. {*} is a pushed button, while { } is a button that is not pushed. ( ) represents a hole for a banana plug. Each patch cord has a banana plug on each end. Plugging a patch cord between separate panels will void the warranty ;-).

@@ -2392,19 +2526,19 @@ - + The machine block diagram - - + +
Figure: RT block diagram
- Figure . The RT Machine block diagram + Figure . The Realizable Machine block diagram -

This section describes the RT Machine organization. Figure shows the major components and their channels of communications. The organization guides designers who will later draft schematics that specify all of the connections.

+

This section describes the Realizable Machine organization. Figure shows the major components and their channels of communications. The organization guides designers who will later draft schematics that specify all of the connections.

Components

@@ -2450,7 +2584,7 @@ List . The units and panels the machine is assembled from -

As described in chapter , which is being used as the architectural template, the executor guides the machine through the initialization, programmed control, and halting stages of execution. The Executor contains the power, reset, and clock logic. It has two modes of execution: the run mode in which the clock runs free, and the single-step mode, in which clock pulses are sent with the push of a button. It also holds the halt state switch bank, and is ready to stop the clock when a halt state is reached. When in the programmed control stage, most of the active control comes from the CPCU.

+

As described in chapter , which is being used as the architectural template, the executor guides the machine through the initialization, programmed control, and halting stages of execution. The Executor contains the power, reset, and clock logic. It has two modes of execution: the run mode in which the clock runs free, and the single-step mode, in which clock pulses are sent with the push of a button. It also holds the halt state switch bank, and is ready to stop the clock when a halt state is reached. When in the programmed control stage, most of the active control comes from the CPCU.

The CPCU contains the current state register q, the Next-State Table, and the Instruction Table.

@@ -2468,14 +2602,14 @@ The tape transport unit, TTU, handles mounting, unmounting, reading, writing, and moving the tape. In this volume, we treat head motion in a relative sense, recognizing that a physical TTU moves the magnetic tape beneath a stationary head rather than displacing the head across the tape. Our architecture allows a system to incorporate multiple TTU modules, each containing its own read buffer, status buffer, and head assembly. Tapes can be dynamically mounted, unmounted, shelved, and transferred between any TTU on the same machine or across distinct systems.

- + Description of a read('s') instruction

The current state register, q, samples and holds on the rising edge of the clock. This then provides a stable state signal to the control panel next-state matrix columns for an entire cycle. The status register, s, is the phase inverse; when enabled it samples on the falling edge of the clock and holds for an entire cycle.

On the rising edge of the clock, the new current state signal ripples through the Instruction Table. As the q register is stored as a one-hot value, this happens very quickly. Rather than placing instructions in the Instruction Table, the decoded values to be used as control signals are programmed directly into the table. Among the decoded outputs is a control signal that enables the status register to sample when the clock falls.

-

A control signal from the decoded instruction coming from the Instruction Table then enables the selected TTU and provides it with its decoded instruction. Part of the decode will be the HU select line, and a signal to tell the HU status register to drive the machine's internal data bus.

+

A control signal from the decoded instruction coming from the Instruction Table then enables the selected TTU and provides it with its decoded instruction. Part of the decode will be the HU select line, and a signal to tell the HU status register to drive the machine’s internal data bus.

Thus the status register in the selected TTU and HU will be driving the data bus, and the machine s register is ready to sample the bus. Then the clock falls, the s register updates, and holds its value until the next time when it is enabled to update and the clock falls.

@@ -2492,92 +2626,150 @@ - - Introduction to Computational Naturalism + + Naturalism - - The Thesis + + Platonism -

In March of 1826 Babbage read a paper to the Royal Society describing a language he had invented for machines.Charles Babbage, "On a Method of Expressing by Signs the Action of Machinery", Philosophical Transactions of the Royal Society of London 116 (1826): 250−265, read 16 March 1826. He called it the mechanical notation. It gave each part of a machine a sign, classified that part as fixed or as moveable, recorded what drove it and what it drove in turn, and set down when each motion occurred relative to the others.Babbage considered it one of his best inventions.Charles Babbage, Passages from the Life of a Philosopher (London: Longman, Green, 1864), 104, where he adds that he doubts machinery of comparable complexity could be contrived without that language or an equivalent one. On its mature form, comprising labeled drawings, timing diagrams, and logic diagrams, see Anthony Hyman, Charles Babbage: Pioneer of the Computer (Oxford: Oxford University Press, 1982), 58.

+

Plato (427−347 BC) spoke of an entity with an independent existence he called a Form. A Form is immutable, non-spatial, and non-temporal, and exists in a realm that has no location, called the Realm of Forms. Examples of Forms include mathematical properties (Equality, Circularity, Squareness, Unity, Doubleness), moral concepts (Justice, Courage, Piety, Temperance), aesthetic ideals (Beauty), physical archetypes (Tableness, Bedness, Treeness), and above them all the Good, which Plato held to be what makes the other Forms knowable. Plato then explains that knowledge is the conceptualization of Forms.

-

Imagine a Universal RT Machine

+

Plato pressed the point with an allegory. Men are chained in a cave facing a wall, and behind them a fire throws the shadows of objects onto the wall before them. Having seen nothing else, they take the shadows for the whole of what there is. One of them is freed, is brought up into the light, and sees the objects themselves; and when he returns to tell the others, they do not believe him. + Plato, Republic VII 514a−520a. +

+

As an illustration of the independent existence of Forms, consider what Plato says about geometers in the Republic. They draw a square and a diagonal, and then they argue about it. But they are not arguing about the figure they drew, and everyone participating in the discussion knows this to be the case. They are not asking whether that line, the one in the sand, is straight, nor whether that corner is square, for it plainly is not, and no conclusion they reach depends upon it. The drawn figure is used as an image of the Square itself, and what they say holds of the Square itself whether or not the drawing was made well, or made at all. + Plato, Republic VI 510c−511a. The passage places mathematics on the third segment of the divided line, above opinion and below the account that would justify its hypotheses. Aristotle grants the same observation about mathematical practice and gives it the opposite explanation, at Metaphysics XIII 1078a17−31. +

-

Hence, while the clerk at the desk with his squares of paper, and a pen, works, he is merely a more general machine, a sort of Universal Machine, programmed by the language of mathematics. As he interprets the math, as a Universal Turing Machine would interpret a program, the para is a parallel machine to the RT Machine, just as Turing's Universal Machine running a program is a parallel machine to the machine with a hard coded controller.

+

When a geometer erases his drawing the theorem is untouched. He draws it again badly and the theorem is untouched. Every square that has ever been drawn could be wiped away and the theorem would still hold, and it held before any of them were drawn.

-

And as mathematics is the language of for the universal machine, it can be interpreted as observations of the running base machine. As when a Universal Turing Machine program says to step a square to the right, if the parallel hard controlled machine would also be moving the head one square to the right. So to, can statements in mathematics be shown to be describing the motions of the RT Machine.

+

The definition of a symbol given in section is Platonic, and was written that way before the matter had been thought through. It began with a symbol, which is an abstraction; it required a representation, such as a Greek letter; and it noted that what appears on the written page is an instance of that representation, of which there can be many. Set that beside the geometers. There is the Square, there is the manner in which a square is to be presented, four sides drawn equal and joined square, and there are the many figures scratched in the sand. The symbol is the Form, the representation is the manner in which the Form is met with, and the instance is the shadow. Three levels in each case, and the middle one is the reason section declines the philosophers’ pair of type and token.

 - - - The lamps + + Naturalism + +

Aristotle (384−322 BC) arrived at Plato’s Academy at seventeen and stayed for twenty years. He was the son of a physician, and he had from the beginning the disposition of a man who wants to see the thing itself. Where Plato was brilliant and intuitive, Aristotle was scholarly and methodical. + Will Buckingham, Douglas Burnham, Peter J. King, John Marenbon, Clive Hill, and Marcus Weeks, The Philosophy Book, Big Ideas Simply Explained (New York: DK, 2011), page 58. + The difference shows in what each took to be the starting point of an inquiry.

+ +

Aristotle rejected the Realm of Forms entirely. His objection was that it explains nothing: to account for the many circles in the world by positing one Circle elsewhere is to double the number of things standing in need of explanation while leaving the original question where it was, and to say that a drawn circle participates in the Circle is, in his words, to speak in poetical metaphors. + Aristotle, Metaphysics I.9 990b−991b, with the remark on poetical metaphors at 991a20−22, and again at XIII.4−5. Aristotle does use the word eidos, ordinarily translated as form, for the structure in virtue of which a thing is what it is. That structure is in the thing and nowhere else, and it is not Plato’s Form under another name. One English word is doing service for two concepts, and only the first is at issue here. + Take away the Realm of Forms, and nothing of the account survives. There is no elsewhere for the ideal circle to be, so there is no original for the drawn one to fall short of, so there are no shadows, so there is no cave to be led out of. The drawn circle is a circle.

+ +

He then turned Plato upside down. For Plato, knowledge descends: one conceptualizes the Forms, and the senses report shadows that mislead more than they inform. For Aristotle, knowledge ascends: perception of particulars is where an inquiry starts, many perceptions leave what is common to them, and the general principle is arrived at from below. + Aristotle, Posterior Analytics II.19, for knowledge beginning in perception and rising by induction to the principles. The Greek term for the operation is epagōgē, rendered as induction. +

+ +

Upon leaving the Academy Aristotle then did what his position obliges a man to do, and went and looked. He dissected, he collected, and he questioned fishermen and beekeepers about what they had seen; his account of the developing chick was got by opening eggs on successive days. + Aristotle, Historia Animalium VI.3 561a for the chick; V.12 541b for the modified arm of the male octopus, which was thought a fable until confirmed in 1857. He was wrong about a great deal, holding the heart to be the seat of thought and the brain an organ for cooling the blood. Darwin, thanking William Ogle in 1882 for a translation of the Parts of Animals, wrote that Linnaeus and Cuvier had been his two gods, but that they were mere schoolboys compared to old Aristotle. Charles Darwin to William Ogle, 22 February 1882, in Francis Darwin, ed., The Life and Letters of Charles Darwin (London: John Murray, 1887), vol. 3, 252. + What he produced from it was a taxonomy: animals sorted by the features they share, the sorting answerable to the specimens and revised whenever a specimen refused its category. So Aristotle is a naturalist in both of the senses this book uses. He held that an account is owed in terms of the world, and he spent his life observing and sorting. The two are not separate facts about him. The second is what the first commits a man to.

+ +

This perspective, that knowledge comes from the categorization of observations of nature rather than, as Plato taught, from the conceptualization of Forms, is the pillar of the philosophy of Naturalism. Where the position and the method are distinguished, the first is called metaphysical naturalism and the second methodological naturalism, and a person can hold the second while remaining quiet about the first. The title of this book follows from both. Computation theory is the result of observing and categorizing what the Realizable Machine of chapter does and can be made to do.

+ - - The lamps + + Number as a natural object -

The machine of chapter carries a row of status lamps on its control panel, and they were drawn there without remark. Recall how they are driven. The alphabet is enumerated so that each of its symbols corresponds to one row of the panel. A head reads a cell, the value arrives in the s register, the status decoder converts that value into a one-hot encoding, and the single asserted line illuminates one row.

+

The Aristotelian account was the working position of the schools for most of two thousand years, and it was still the working position when the mathematics that mattered began to change. In the seventeenth century the naturalist reading was held plainly, and held by the people doing the work. Thomas Hobbes (1588−1679) made geometry a science of bodies and of the motions that generate them. + Thomas Hobbes, De Corpore (London, 1655), Part II. He defended the position through a public quarrel with John Wallis that ran over twenty years; see Douglas M. Jesseph, Squaring the Circle: The War between Hobbes and Wallis (Chicago: University of Chicago Press, 1999). + Isaac Barrow (1630−1677) argued in his Cambridge lectures that mathematical magnitudes are abstracted from the magnitudes of physical bodies and have no other source. + Isaac Barrow, Lectiones Mathematicae, delivered 1664−1666 and published (London, 1683), Lectures I−VII. Barrow held the Lucasian chair before Newton, who succeeded him in 1669. +

+ +

Newton (1642−1727) wrote in the preface to the Principia that geometry is founded in mechanical practice and is nothing other than the part of universal mechanics which measures accurately. + Isaac Newton, Philosophiæ Naturalis Principia Mathematica (London, 1687), Author’s Preface. The remark is made in passing, as something the reader will grant, which is itself evidence of how the matter then stood. + A man who says that geometry is a branch of mechanics is saying that the forms are in the machinery. It is worth recalling that the vocabulary of the period made the claim easy to state. The study of the physical world was called natural philosophy, and Newton’s book of 1687 is the Mathematical Principles of Natural Philosophy. The word scientist was not coined until 1834. + William Whewell coined it in an unsigned review of Mary Somerville, On the Connexion of the Physical Sciences, Quarterly Review 51 (1834): 54−68, and put it into print under his own name in The Philosophy of the Inductive Sciences (London: Parker, 1840), vol. 1, cxiii. The older term survives in the chairs of natural philosophy at the Scottish universities. + The vocabulary changed before the claim was settled.

+ +

The eighteenth century turned against the position from two directions at once. George Berkeley (1685−1753) denied that there are abstract general ideas at all, and then turned that denial on the calculus in The Analyst of 1734, where the vanishing increments of the new analysis are asked what they are and found to be neither finite quantities, nor nothing, but the ghosts of departed quantities. + George Berkeley, A Treatise Concerning the Principles of Human Knowledge (Dublin, 1710), Introduction §§7−25, against abstract ideas; The Analyst (London, 1734), §35 for the phrase. Berkeley’s target was the reasoning and not the results, and the objection stood unanswered until the rigorization of the following century. + The attack was fatal to complacency and did not restore the naturalist account, since Berkeley’s own position placed the objects in the mind rather than in the world. David Hume (1711−1776) then divided all inquiry into relations of ideas and matters of fact, and placed mathematics squarely among the former, where nothing observed bears on it. + David Hume, An Enquiry Concerning Human Understanding (London, 1748), §IV part 1. + An empiricist had thus removed mathematics from the reach of experience.

+ + + + The turn inward -

One-hot means one line asserted and no others. It is not a policy the machine follows but a fact about how the decoder is cut: the lines are driven from a complete decode of the register, so exactly one of them is high for any value the register can hold. There is a lamp for each symbol, no lamp for anything else, and never two lamps at once.

+

Immanuel Kant (1724−1804) settled the matter for the next century in the Critique of Pure Reason of 1781. Mathematical judgements are necessary, so they are not got from experience; they are also not empty, since seven and five taken together yield a twelve that no analysis of the concepts of seven, five, and sum will produce. They are therefore synthetic and a priori both, and what makes them possible is the pure intuition of space and time, which is a form contributed by the knowing subject rather than a feature found in the world. + Immanuel Kant, Kritik der reinen Vernunft (Riga: Hartknoch, 1781; second edition 1787), Introduction B14−B17 for the arithmetical example, and the Transcendental Aesthetic for space and time as forms of intuition. +

-

Nor is any judgment exercised anywhere along that path. The objection presents itself early, so it is worth closing early. If a lamp reports a voltage, and voltage is continuous, then somewhere a threshold was chosen and a decision made, and a symbol resting on a decision was imposed by whoever made it. But the machine holds no thresholding device. Take the CMOS inverter. A high input turns the N device hard on and the P device off, and the output is pulled to ground; a low input does the reverse. Through the middle region its gain is far above unity, so an input that wanders toward the middle is driven away from it rather than adjudicated. Noise margins are not decision points. They are the width of the region the gate's own gain repairs. The lamp is not deciding whether to illuminate; it is commanded to, or it is held dark, and no third behavior is available to it.There are devices that do decide, and they are worth naming so as not to be mistaken for the general case. A communications receiver samples a line and hands the sample to a decider. A memory array develops a small differential on its bit lines and hands it to a sense amplifier. Both restore a signal degraded past what gain alone recovers, and both sit at the periphery, on the boundary between the machine and what is not yet digital. That boundary is where symbols enter and leave. It is not what the machine is made of.

+

It is worth being exact about what this does. Kant does not deny that mathematics is grounded. He relocates the ground, from the world to the structure of the mind that apprehends the world, and in doing so he answers the question the naturalist had been trying to answer, and answers it better than the naturalist then could. Geometry is necessary because space is our form of outer intuition, and it applies to everything we can experience because we can experience nothing outside that form. For a hundred and thirty years the position looked unassailable.

-

So the lamp is lit if and only if the symbol is present, and the same decoder line does both jobs. A person watching the panel is not reading an instrument that formed an opinion about the machine. He is watching the machine.

+

What unmade it was not an argument but a discovery. Nikolai Lobachevsky (1792−1856) published a geometry in which the parallel postulate fails, in 1829; János Bolyai (1802−1860) published another in 1832; Carl Friedrich Gauss (1777−1855) had reached the same results earlier and withheld them. Bernhard Riemann (1826−1866) generalized the question in 1854, and Eugenio Beltrami (1835−1900), whose model is discussed in chapter , showed in 1868 that the new geometry is consistent if the old one is. + N. I. Lobachevsky, “On the Principles of Geometry,” Kazan Messenger, 1829−1830; János Bolyai, appendix to Farkas Bolyai, Tentamen (Maros-Vásárhely, 1832); Bernhard Riemann, “Über die Hypothesen, welche der Geometrie zu Grunde liegen,” delivered 1854, published Abhandlungen der Königlichen Gesellschaft der Wissenschaften zu Göttingen 13 (1868): 133−152; Eugenio Beltrami, “Saggio di interpretazione della geometria non-euclidea,” Giornale di Matematiche 6 (1868): 284−312. + If there are several consistent geometries and physical space satisfies at most one of them, then geometry is not the science of space, and the necessity Kant had explained is not there to be explained. The best example either party had of mathematics answering to the world had been taken off the table, and it was taken off by mathematicians rather than by philosophers.

 - - One report, carried across + + The last stand and the demolition -

The machine is in single-step mode, and there are five symbols in its alphabet and so five status lamps. The operator pushes the step button and reports what he saw. What follows is that report, rewritten six times.

+

The fullest naturalist account of number ever written appeared in the middle of this, and is therefore later than most readers expect. John Stuart Mill (1806−1873) argued in A System of Logic of 1843 that the truths of arithmetic are inductive generalizations from experience, of the same kind and standing as the generalizations of any other science. Two and one make three is a fact about collections of objects, learned the way facts about collections of objects are learned, and it is necessary only in the sense that nothing has ever contradicted it. + John Stuart Mill, A System of Logic, Ratiocinative and Inductive (London: Parker, 1843), Book II, chaps. 5−6, and Book III, chap. 24. +

- - -
    -

  1. I pushed the step button, and the third lamp from the left came on while the others stayed dark.

    -

    The report as spoken. It carries a person, an act, a moment, and a direction of gaze.

    2. +

    Gottlob Frege (1848−1925) destroyed the position in the Grundlagen der Arithmetik of 1884, and did so at leisure and by name. If a number is a property of a heap of things, then a single pack of cards is one, and fifty-two, and four, depending on what one has chosen to count, so the number is not in the heap. If arithmetic rests on what has been observed, then its necessity is the necessity of a habit, and a sufficiently strange experience would revise it. And if numbers are ideas, then they are episodes in somebody’s head, and my two and your two are two different objects, and nothing has been said about number at all. This last charge Frege named psychologism, and it became the period’s term of abuse. + Gottlob Frege, Die Grundlagen der Arithmetik (Breslau: Koebner, 1884), Introduction and §§7−10 against Mill, §§21−25 against number as a property of external things, and §§26−27 against number as an idea. Frege pressed the charge again in his review of Husserl, Zeitschrift für Philosophie und philosophische Kritik 103 (1894): 313−332, with enough effect that Husserl abandoned the position. +

    + +

    The demolition was thorough and it was, on its own terms, correct. Every one of those objections is answered in this book, and none of them is answered by disputing what Frege said about Mill. They are answered by giving numbers a source that Mill did not have available to him, which is the subject of chapter , and by an account of what a symbol is that makes my two and your two the same object, which is the subject of chapter . Mill was arguing from heaps of pebbles. There was no other mechanism then to argue from.

    -

  2. Push of the step button: lamp ℓ_2 lit, lamps ℓ_0, ℓ_1, ℓ_3, ℓ_4 dark.

    -

    The lamps have been given names according to the row each occupies. Naming is not yet a claim, and no lamp has moved.

    3. +

    After 1884 the field belonged to programmes that dispensed with the world. Logicism derived mathematics from logic; formalism treated it as the manipulation of marks under stated rules; intuitionism grounded it in mental construction. Richard Dedekind (1831−1916) wrote in 1888 that numbers are free creations of the human mind, and the remark drew no objection. + Richard Dedekind, Was sind und was sollen die Zahlen? (Braunschweig: Vieweg, 1888), preface. Leopold Kronecker’s better-known remark that God made the whole numbers and all else is the work of man is reported by Heinrich Weber, “Leopold Kronecker,” Jahresbericht der Deutschen Mathematiker-Vereinigung 2 (1893): 5−31, at 19. + Formalism in particular gained by not looking at the world, since a formal system is answerable to its own rules alone and is very much the easier to study for it. The crisis recounted in chapter was fought entirely among these three, and no party to it proposed consulting an apparatus.

    -

  3. \text{lit}(ℓ_2) holds, and \text{lit}(ℓ_0), \text{lit}(ℓ_1), \text{lit}(ℓ_3), \text{lit}(ℓ_4) fail.

    -

    The word lit has been written in front of what it is said of, and the five lamps have been taken together as what it is said of. A reader who wants the conventional names for these two moves will find them in chapter under predicate and domain. The lamps are unaffected by being named that.

    4. +

    It is worth recording what was available and went unused. In March of 1826 Babbage read a paper to the Royal Society describing a language he had invented for machines. + Charles Babbage, “On a Method of Expressing by Signs the Action of Machinery,” Philosophical Transactions of the Royal Society of London 116 (1826): 250−265, read 16 March 1826. + He called it the mechanical notation. It gave each part of a machine a sign, classified that part as fixed or as moveable, recorded what drove it and what it drove in turn, and set down when each motion occurred relative to the others. Babbage considered it one of his best inventions. + Charles Babbage, Passages from the Life of a Philosopher (London: Longman, Green, 1864), 104, where he adds that he doubts machinery of comparable complexity could be contrived without that language or an equivalent one. On its mature form, comprising labeled drawings, timing diagrams, and logic diagrams, see Anthony Hyman, Charles Babbage: Pioneer of the Computer (Oxford: Oxford University Press, 1982), 58. +

    -

  4. \text{bind}(s, σ_2)

    -

    Lamp ℓ_i is driven by the decoder line for σ_i, and that line is asserted exactly when the s register holds σ_i. So \text{lit}(ℓ_i) and the status symbol being σ_i are one report and not two, and the shorter phrasing has been kept. The substitution was licensed by the wiring, not by the operator.

    5. +

    So a notation whose subject matter is an apparatus existed sixty years before Frege wrote, and the machine it was invented to describe was on the drawing board in London while the foundational crisis was being fought in Germany. The two never met. Mathematics had, in that period, everything it needed to be grounded in a machine, and instead spent fifty years proving there was nothing to look at.

    -

  5. The operator pushes again. Each push is a separate occasion, and to speak of more than one at a time the occasion is written as an argument: \text{lit}(ℓ_2, c).

    -

    This is where the act and the moment went. The operator, the button, and the afternoon are not discarded; they are what distinguishes one occasion from another. The conventional word for such an occasion is evaluation context.

    6. +

    It is therefore worth noticing what the paper that closed the episode opens with. Alan Turing, asked to settle a question in the foundations of logic, begins by describing a man sitting at a desk with paper and a pencil, and asks what such a man can be got to do.

    + -

  6. ∀c \, ∃i \, \big( \text{lit}(ℓ_i, c) ∧ ∀j \, ( \text{lit}(ℓ_j, c) → j = i ) \big)

    -

    Exactly one lamp is lit, whichever push is taken. What licenses the ∀ is the decoder, which asserts one line and no more for any value the register can hold. The operator does not arrive at it by exhausting the pushes.

    7. -
- List . An operator's report carried into the predicate calculus - + + The return + +

Naturalism came back into philosophy in the second half of the twentieth century, chiefly through Quine, who held that epistemology is a chapter of natural science rather than a tribunal sitting above it, and that our commitment to mathematical objects stands or falls with the theories that cannot be stated without them. It did not come back into the foundations of mathematics in any form that touched practice. + W. V. Quine, “Epistemology Naturalized,” in Ontological Relativity and Other Essays (New York: Columbia University Press, 1969), 69−90. See also Imre Lakatos, Proofs and Refutations (Cambridge: Cambridge University Press, 1976), on mathematics as it is actually done, by conjecture, counterexample, and repair; Hilary Putnam, “What is Mathematical Truth?” Historia Mathematica 2 (1975): 529−543, which names the position quasi-empiricism; and Penelope Maddy, Naturalism in Mathematics (Oxford: Clarendon Press, 1997) and Second Philosophy: A Naturalistic Method (Oxford: Oxford University Press, 2007). +

-

The last line is a sentence of the first-order predicate calculus, built by the formation rules and by nothing else. It is also a statement about lamps, true of a machine that was wired before anyone wrote it down, and it would have been true of the machine while it was still a drawing. There is one thing here being described in two notations, the longer of which was abandoned at line 2.

+

The closest of these to the present book is Philip Kitcher’s, which gives mathematical knowledge as knowledge of operations that an idealized agent performs, collecting, ordering, and matching, rather than of objects standing apart. + Philip Kitcher, The Nature of Mathematical Knowledge (New York: Oxford University Press, 1983). + The distance is still the whole of the distance. His agent is idealized, which is to say stipulated, and what it can do is settled by the philosopher who describes it. The machine in these pages is not stipulated. It is built, and what it does when it runs is a question with an answer that nobody supplies.

 - - Where the generality comes from + + What the word means here -

Read the ladder again looking for the step at which the account left the machine. There is not one. Line 2 names, line 3 puts a word in front of what it is said of, line 4 substitutes one phrase for another on the authority of a wire, line 5 admits there is more than one push, and line 6 says every. A reader who wants to locate the moment at which mathematics took over from engineering will not find it, and that absence is the argument of this chapter.

+

Two things the word does not mean in this book. It does not mean natural computing, the field that draws its methods from living systems, from molecules, swarms, and neurons. The machine in these pages is a tape and a controller and nothing about it is borrowed from anything alive. And Natural does not qualify a number. What mathematics calls the natural numbers are Peano Numbers here, for the reason given in section , which leaves the adjective to the philosophy alone.

-

Line 6 repays a second look, because a universal claim from a man who has pushed a button a few times looks like an overreach. It would be, if the tally were what licensed it. It is not. The decoder asserts one line and no more, for every value the register is capable of holding, and that is a fact about how the part was cut. The operator could read it off the wiring diagram without pushing the button at all, which is what Babbage did for an engine that had no buttons to push. The generality is in the construction. The notation records it.

+

What it does mean is the position reviewed above, held about mathematics, and pursued by the method the older sense of the word names. Computational Naturalism is the thesis that mathematics is a taxonomy of observations of a machine. The conditions such a machine must satisfy are set out in chapter , the machine itself is built in chapter , and the taxonomy occupies most of what follows that.

-

This also answers where the operator went. The final sentence mentions no person and no afternoon, and it does not follow that these were discarded as unreal. They were absorbed into c. Every occasion is a particular push, the quantifier ranges over those pushes and over nothing else, and the generality of the sentence is not achieved by rising above the machine but by declining to write its occasions out one at a time.

- +

Consider a Turing Machine program that prints the character s, loops back, and prints it again without end. It cannot be run to completion. It can nonetheless be analyzed, discussed, and reasoned about, as we are doing at this moment. Here then is the situation in miniature: there are things the first order cannot reach, and a language standing above it in which those very things are said.

- - Reading the ladder downward +

Accordingly the Realm of Forms is not a separate realm. It is the tower of languages standing above the machine, each layer of it there to speak of the holes in the layer beneath. Those languages become separated from the machine that gave rise to them, so the base machine never need be run. What builds the tower is exhaustion. Each layer runs out of what it can say, and the running out is what makes the next layer necessary. The layers are the orders of analysis, defined in section .

-

A ladder can be climbed in either direction. Hand the final sentence to a person who has never seen the machine and ask what would have to be the case for it to hold. He arrives back at a panel on which exactly one lamp lights per step, or at a wiring diagram from which that follows. The sentence carries its own conditions of redemption, and they are conditions on an apparatus.

+

Both parties are therefore granted what they asked for. Aristotle is granted that the account begins in an object one can point at and that nothing enters from outside it. Plato is granted a realm above the particulars whose inhabitants are not to be found by running anything. What is denied is the separation, and only that. The tower has a floor, the floor is an apparatus, and the orders are how one climbs.

-

That is the thesis, and the chapters ahead are spent making the descent for one mathematical structure after another: the areas and partitions of a tape, the metrics over them, the Peano Numbers, the addresses, and the symbol itself considered more closely than it has been here. In each case the question asked is the one Babbage's signs were built to answer. What machine, and what will it do?

+

Stephen Kleene opens his Introduction to Metamathematics with a flock of four sheep and a grove of four trees, and observes that one can pair them off, a sheep to a tree, without counting either. + Stephen Cole Kleene, Introduction to Metamathematics (Amsterdam: North-Holland, 1952), 3. + The passage is there to introduce Cantor, and it does something else on the way. It appeals to Naturalism as a foundation, it leverages the reader’s intuition, and it makes the implicit point that mathematics has a purpose. Nobody objects to the sheep.

-

One detail of his notation is worth carrying forward. The first thing his signs recorded of any part was whether it was fixed or moveable. A reader who has been through chapter will recognize the pair, since the same two words are the conventional ones for sorting symbols by whether what they are bound to holds across every occasion or varies between them. The classification did not travel from mathematics to the machine. It was recorded of gears and levers first.

- +

The sheep are then dismissed and the strings arrive. This is worth pausing over, because formalism was sold as the cure for Platonism. No abstract objects and no separate realm, only marks on paper and rules for replacing them. But consider the marks. A formal string never smudges, never exhausts the paper, and costs nothing to write. Two occurrences of a symbol are perfectly identical, on every page, in every century. Those are Forms. Formalism did not empty the Realm. It evicted the numbers and moved the symbols in.

+

What this book proposes is that the appeal to the sheep be taken seriously rather than used and set aside, and that the flock be replaced by an apparatus whose behaviour can be established rather than assumed. The strings then cost something, the alphabet is finite because somebody had to build the decoder, and emptiness is a property of a cell rather than a character written in it. + The category error in treating emptiness as a symbol is taken up in section . The cost of an alphabet, measured in the size of the controller that must decode it, is taken up in section . +

+ @@ -2697,7 +2889,7 @@

- When an input tape is provided as a general mathematical object, either decreed by definition or perhaps abstracted from 'what a Turing Machine computation would produce in the limit of step-count', then the input can be either finite or infinite. + When an input tape is provided as a general mathematical object, either decreed by definition or perhaps abstracted from ‘what a Turing Machine computation would produce in the limit of step-count’, then the input can be either finite or infinite.

@@ -2714,7 +2906,7 @@

- In contrast, out-of-band control communicates structural information through a strictly separate channel or by utilizing symbols definitively excluded from the programmer visible data alphabet. The rightmost tape marker is an out-of-band mechanism because it utilizes an expanded hardware tape alphabet strictly reserved for machine management, guaranteeing it can never be conflated with the user's data. Modern architectures often lack the luxury of inventing new symbols to serve as control rather than data. Another out-of-band signaling technique is to structure the data into channels; such structure is called formatting. We find formatting on hard drives, in frame based and packet based communication channels, and in data structures. + In contrast, out-of-band control communicates structural information through a strictly separate channel or by utilizing symbols definitively excluded from the programmer visible data alphabet. The rightmost tape marker is an out-of-band mechanism because it utilizes an expanded hardware tape alphabet strictly reserved for machine management, guaranteeing it can never be conflated with the user’s data. Modern architectures often lack the luxury of inventing new symbols to serve as control rather than data. Another out-of-band signaling technique is to structure the data into channels; such structure is called formatting. We find formatting on hard drives, in frame based and packet based communication channels, and in data structures.

@@ -2744,7 +2936,7 @@

- There is a small problem, in that the left-of-leftmost symbol was stipulated as being separate from the alphabet, and only alphabet symbols and the empty symbol can be written to the tape. There are a couple of possible solutions. One solution is to add a symbol to the alphabet that is also taken to mean 'left of leftmost', say gasket-left-of-leftmost. Then machines that make use of gaskets can recognize this new symbol for its meaning. Another alternative, that will be used for the remainder of our discussion, is to define a separate control alphabet of symbols that are distinct from data alphabet symbols, but can also be written and read from the tape. We will add to this alphabet, empty, left-of-leftmost, and a new control symbol used by gaskets,right-of-rightmost. + There is a small problem, in that the left-of-leftmost symbol was stipulated as being separate from the alphabet, and only alphabet symbols and the empty symbol can be written to the tape. There are a couple of possible solutions. One solution is to add a symbol to the alphabet that is also taken to mean ‘left of leftmost’, say gasket-left-of-leftmost. Then machines that make use of gaskets can recognize this new symbol for its meaning. Another alternative, that will be used for the remainder of our discussion, is to define a separate control alphabet of symbols that are distinct from data alphabet symbols, but can also be written and read from the tape. We will add to this alphabet, empty, left-of-leftmost, and a new control symbol used by gaskets,right-of-rightmost.

Now suppose defining a Turing Machine that initially has the head on the leftmost cell of a specific area. When step left is called, the tape transport immediately returns the left of leftmost symbol, independent of whether the area is actually at the physical left end of the tape or not. @@ -2804,7 +2996,7 @@

As we noted in section , an area has two distinguishing features, being that it has a leftmost cell, and a rightmost cell. That definition is topological. If we start with the leftmost cell of a tape, we are either already on the leftmost cell of a defined area, or we can step right to find it. At the point of finding it we know this leftmost cell is part of the area, then we are either already on the rightmost cell, or we can continue to step right on cells in the area until we find the rightmost cell in the area. The right neighbor of the rightmost cell in the area, and all cells to that right of that, are excluded from the area.

-

With addresses we can now define an area with two addresses, two Peano Numbers, the address of the leftmost cell, and that of the rightmost cell. All Peano Numbers greater than or equal to the address of the leftmost cell, or less than or equal to the address of the rightmost cell, are addresses of cells that are in the area. This feels more satisfactory for most of us, as now we are talking about arithmetic rather than graph topology. Though should the topology of the tape be generalized, this could become limiting. It reminds me of Frege's admonition that perhaps math should be built on top of geometry. +

With addresses we can now define an area with two addresses, two Peano Numbers, the address of the leftmost cell, and that of the rightmost cell. All Peano Numbers greater than or equal to the address of the leftmost cell, or less than or equal to the address of the rightmost cell, are addresses of cells that are in the area. This feels more satisfactory for most of us, as now we are talking about arithmetic rather than graph topology. Though should the topology of the tape be generalized, this could become limiting. It reminds me of Frege’s admonition that perhaps math should be built on top of geometry.

@@ -2812,7 +3004,7 @@ Distance and extent -

The distance between two cells is the absolute difference in their addresses. When we talk about the distance between the leftmost cell of an area and the rightmost cell of an area, we call this number the area's extent.

+

The distance between two cells is the absolute difference in their addresses. When we talk about the distance between the leftmost cell of an area and the rightmost cell of an area, we call this number the area’s extent.

In an analogous manner to defining an address space for a tape, we can define an address space for an area. Accordingly, the leftmost cell of the area is assigned address zero, and the address increments cell by cell going to the right. A finite area will have a finite address space, with the address of the rightmost cell being the largest address in the address space.

@@ -2825,10 +3017,10 @@ Length -

If cells in an area are transacted, the cost of the area is the cost of a cell multiplied by the count of cells. Vincent Atanasoff probably found himself needing to know such a count when ordering capacitors. The count of cells in an area is also known as the area's length. We will use the Greek symbol Ï¡, pronounced as sampi, to refer to the length. The length of an area, the count of its cells, will always be one greater than its extent, Ï¡ = \omega + 1. This symbol makes sense here, as the Greek number system fell short of letters, so the symbol Ï¡ was tacked on to the end of the alphabet, but did not belong to the alphabet. +

If cells in an area are transacted, the cost of the area is the cost of a cell multiplied by the count of cells. Vincent Atanasoff probably found himself needing to know such a count when ordering capacitors. The count of cells in an area is also known as the area’s length. We will use the Greek symbol ϡ, pronounced as sampi, to refer to the length. The length of an area, the count of its cells, will always be one greater than its extent, ϡ = \omega + 1. This symbol makes sense here, as the Greek number system fell short of letters, so the symbol ϡ was tacked on to the end of the alphabet, but did not belong to the alphabet.

-

The count of cells in an area, the length of an area, and the cardinality of the address space for an area are all the same number, Ï¡. The extent of an area, \omega, is an address in an area's address space, whereas the cardinality of an area, Ï¡, falls outside it. This has implications. Because extent is an address, extent and addresses can always use the same number representation. In contrast, there is no such guarantee for cardinality, Ï¡. Take for example an area that contains 256 cells. The addresses run from zero to 255, and all can be represented with an 8 bit binary number. However, the number 256 requires 9 bits, and thus would not fit in an 8 bit address register. This one of the reasons that code will have fewer end-case problems when expressing the size of objects with extents, rather than with lengths. +

The count of cells in an area, the length of an area, and the cardinality of the address space for an area are all the same number, ϡ. The extent of an area, \omega, is an address in an area’s address space, whereas the cardinality of an area, ϡ, falls outside it. This has implications. Because extent is an address, extent and addresses can always use the same number representation. In contrast, there is no such guarantee for cardinality, ϡ. Take for example an area that contains 256 cells. The addresses run from zero to 255, and all can be represented with an 8 bit binary number. However, the number 256 requires 9 bits, and thus would not fit in an 8 bit address register. This one of the reasons that code will have fewer end-case problems when expressing the size of objects with extents, rather than with lengths.

@@ -2844,7 +3036,7 @@
    -
  1. We set Turing Machine P with its head on the leftmost cell of an area. We mount the initial tape, '·', on the A machine.
    2. +
  2. We set Turing Machine P with its head on the leftmost cell of an area. We mount the initial tape, ‘·’, on the A machine.
  3. We step P and simultaneously run the A machine. Stepping stops when P reaches the rightmost cell of the area. At this point, the tape on the A machine holds the address space extent.
  4. The A machine is run an additional time. The output on the tape is defined to be the cardinality of the address space, aka the cardinality.
@@ -2893,7 +3085,7 @@

The resolution lies in the computational reality of Step 2. For an area that is open on the right, the stepping of machine P never halts. Because Step 2 never terminates, Step 3 is never executed. The A machine never runs that final, additional time. Therefore, the cardinality of an open area is never actually produced by the machine. In the language of Computational Naturalism, Lemma 2 is false for an infinite area; the cardinality of an open on the right area is excluded from being a Peano Number because a Peano Number Machine cannot reach it in the first-order.

-

So then can we add a property to cardinality, such that a second-order analysis could use this property to continue downstream analysis? In short we could say that cardinality has no first-order value, but it has a second-order one. This is analogous to inventing a new type of number, analogous to a complex number with a second component. I.e., there is no 'real' solution, but there is an 'imaginary' one. Or analogous to error algebra, where a number value is replaced with a rule on how to handle downstream operations when it is given as an input. +

So then can we add a property to cardinality, such that a second-order analysis could use this property to continue downstream analysis? In short we could say that cardinality has no first-order value, but it has a second-order one. This is analogous to inventing a new type of number, analogous to a complex number with a second component. I.e., there is no ‘real’ solution, but there is an ‘imaginary’ one. Or analogous to error algebra, where a number value is replaced with a rule on how to handle downstream operations when it is given as an input.

Such a value would be a new Turing Machine, one that composes a call to the never halting Peano Number machine followed by an increment operation. It cannot be run, but it perfectly explains the situation to an analyst. Perhaps we name this machine \aleph_0. @@ -2944,27 +3136,27 @@ Unary representation

- A person can define a Turing Machine that is identical to the recursive definition of Peano Numbers as given by Peano. Giuseppe Peano, Arithmetices principia, nova methodo exposita (Turin: Fratres Bocca, 1889). If a person were to run the Peano Number Machine and observe as it writes to the tape, that person would watch as the Peano Numbers are printed one after another: '·s·ss·sss·ssss· ...'. Here '·' represents zero, and 's·' represents one, etc. Because the Peano Number Machine never halts, the machine cannot be used to initialize a tape, but an analyzer can evaluate the machine to make conclusions about what it would write if it were run. + A person can define a Turing Machine that is identical to the recursive definition of Peano Numbers as given by Peano. Giuseppe Peano, Arithmetices principia, nova methodo exposita (Turin: Fratres Bocca, 1889). If a person were to run the Peano Number Machine and observe as it writes to the tape, that person would watch as the Peano Numbers are printed one after another: ‘·s·ss·sss·ssss· ...’. Here ‘·’ represents zero, and ‘s·’ represents one, etc. Because the Peano Number Machine never halts, the machine cannot be used to initialize a tape, but an analyzer can evaluate the machine to make conclusions about what it would write if it were run.

- In contrast, a function extension version of the Peano Number sequence generator can be run. The function extension machine would be given as input a prior function extension result, or an initial empty tape. It would then modify the tape to contain the next Peano Number, as per the sequence that the Peano Number machine would write, if it could be run. This is accomplished through two subroutine calls: find_empty and increment. The find_empty machine checks the symbol under the head. Upon finding it is not the empty symbol, the machine steps right and checks the next cell, repeating until it finds the empty symbol, whereupon it halts. The increment machine then writes an 's' onto the tape and halts. Recurrent calls to the Peano Number extender then produce a sequence of result tapes: [], [s], [ss], …. Similarly, a programmer can write a machine called decrement, though the programmer must note that decrementing can return the left-of-leftmost symbol. + In contrast, a function extension version of the Peano Number sequence generator can be run. The function extension machine would be given as input a prior function extension result, or an initial empty tape. It would then modify the tape to contain the next Peano Number, as per the sequence that the Peano Number machine would write, if it could be run. This is accomplished through two subroutine calls: find_empty and increment. The find_empty machine checks the symbol under the head. Upon finding it is not the empty symbol, the machine steps right and checks the next cell, repeating until it finds the empty symbol, whereupon it halts. The increment machine then writes an ‘s’ onto the tape and halts. Recurrent calls to the Peano Number extender then produce a sequence of result tapes: [], [s], [ss], …. Similarly, a programmer can write a machine called decrement, though the programmer must note that decrementing can return the left-of-leftmost symbol.

- To say that Peano Number A is smaller than Peano Number B is to say that A would occur on the Peano Number Machine tape to the left of B, if the machine were run. Conversely, if B were said to be greater than A, that would mean B occurs further to the right. It is a simple matter for a programmer to write a decider machine for this. The decider is given two input tapes for the two numbers to compare, and it keeps a head on each. It then steps forward until neither head has an 's' under it. If empty symbols are found simultaneously on both tapes, then the two numbers are equal; otherwise, the number with an empty symbol under the head is the lesser number. + To say that Peano Number A is smaller than Peano Number B is to say that A would occur on the Peano Number Machine tape to the left of B, if the machine were run. Conversely, if B were said to be greater than A, that would mean B occurs further to the right. It is a simple matter for a programmer to write a decider machine for this. The decider is given two input tapes for the two numbers to compare, and it keeps a head on each. It then steps forward until neither head has an ‘s’ under it. If empty symbols are found simultaneously on both tapes, then the two numbers are equal; otherwise, the number with an empty symbol under the head is the lesser number.

- As such, a person can assign a Peano Number to each cell of a given tape by using a mechanical procedure. Given a machine, say P, and an address machine, say A_P, each time P is stepped right, a call is made to run increment on A_P. Similarly, each time P is stepped left, a call is made to run decrement on A_P. In this manner the tape on A_P will always hold the address that machine P's head is on. + As such, a person can assign a Peano Number to each cell of a given tape by using a mechanical procedure. Given a machine, say P, and an address machine, say A_P, each time P is stepped right, a call is made to run increment on A_P. Similarly, each time P is stepped left, a call is made to run decrement on A_P. In this manner the tape on A_P will always hold the address that machine P’s head is on.

- The Peano Number found on A_P is then called the address for the cell that machine P's head is on. As each increment and decrement of the address is a constant-time operation, keeping the address of the cell the head is on is computationally inconsequential. + The Peano Number found on A_P is then called the address for the cell that machine P’s head is on. As each increment and decrement of the address is a constant-time operation, keeping the address of the cell the head is on is computationally inconsequential.

- An address space is a set of addresses for contiguous cells. The tape's address space is the set of addresses for all the cells on the tape. Typically the address of 0 is given to the leftmost among the contiguous cells. + An address space is a set of addresses for contiguous cells. The tape’s address space is the set of addresses for all the cells on the tape. Typically the address of 0 is given to the leftmost among the contiguous cells.

@@ -3096,7 +3288,7 @@ The Computability of Length and Unbounded Zeros -

Consider the proposition of representing Peano Numbers with an unbounded sequence of leftward-trailing zero symbols, extending from the leftmost nonzero digit. If a Turing Machine attempts to evaluate the length of this number, or append a digit to the left of its most-significant digit, the machine encounters a structural failure. Any algorithm that starts at the right of the sequence (the least-significant digit) and scans leftward in search of the last non-zero digit can never be sure at any step that there isn't another non-zero digit further to the left, as those members of the sequence have not yet been examined.

+

Consider the proposition of representing Peano Numbers with an unbounded sequence of leftward-trailing zero symbols, extending from the leftmost nonzero digit. If a Turing Machine attempts to evaluate the length of this number, or append a digit to the left of its most-significant digit, the machine encounters a structural failure. Any algorithm that starts at the right of the sequence (the least-significant digit) and scans leftward in search of the last non-zero digit can never be sure at any step that there isn’t another non-zero digit further to the left, as those members of the sequence have not yet been examined.

Thus, an append function can never know where to write the appended digit, and a length function can never know when to end the count. Because these functions cannot be computed locally on a tape initialized with unbounded zeros, the viable Arabic Representation must strictly be a finite, growing sequence rather than an unbounded string of zeros. If an algorithm attempts to access an index greater than the extent of this finite sequence, the access function fails. This failure is itself a form of meta-information—information about the structure of the representation rather than the number it encodes.

@@ -3106,7 +3298,7 @@ The Representation of Zero and Structural Emptiness

In the growing Arabic representation, counting acts as a mechanical mechanism. In the case of zero, before the first increment, the mechanism has not yet encountered a carry-in. At this stage, no digits have materialized in the representation. Any attempt to retrieve a digit in this state results in an access violation.

-

This reveals a fundamental property of Arabic Representation: it encodes both the sequence of digit symbols (which map to a Peano Number) and the sequence's length (which exists at a meta level, governing computational operations). The representation of zero exposes a limitation. At zero, the representation retains length information necessary for computation but lacks an explicit numerical mapping to zero itself.

+

This reveals a fundamental property of Arabic Representation: it encodes both the sequence of digit symbols (which map to a Peano Number) and the sequence’s length (which exists at a meta level, governing computational operations). The representation of zero exposes a limitation. At zero, the representation retains length information necessary for computation but lacks an explicit numerical mapping to zero itself.

Structurally, this forces a design choice between two options:

@@ -3131,7 +3323,7 @@ a = \sum_{i=0}^n x_i \cdot ten^i -

Here a is a numeric value, so it is written in lower case. Each x_i is the ith component of the sequence X. The value x_i is also known as the 'ith digit' of the number. The value ten is called the base or the radix of the number. It is fortunate that ten is so well known that it has a name, because otherwise an author is tempted to write the base while using the exact representation that he is trying to define. Although this function gives numeric meaning to our digit sequences, performing the suggested computation is pointless. The result is a number, and a person must represent that number, resulting in the exact sequence that was given as the input to the function in the first place.

+

Here a is a numeric value, so it is written in lower case. Each x_i is the ith component of the sequence X. The value x_i is also known as the ‘ith digit’ of the number. The value ten is called the base or the radix of the number. It is fortunate that ten is so well known that it has a name, because otherwise an author is tempted to write the base while using the exact representation that he is trying to define. Although this function gives numeric meaning to our digit sequences, performing the suggested computation is pointless. The result is a number, and a person must represent that number, resulting in the exact sequence that was given as the input to the function in the first place.

In many contexts, a person drops the container notation without causing ambiguity. Conventionally, dropping the notation reverses the order of the components. The sequence from the prior example, X = [7, 8, 9], becomes x = 987. To be clear, note that this number is 13 away from a thousand.

@@ -3142,7 +3334,7 @@ Meaning of the word digital -

The information about ENIAC in this chapter is found in a 1947 IRE article, "Electronic Computing Circuits of the ENIAC" by Arthur W. Burks Arthur W. Burks, "Electronic Computing Circuits of the ENIAC," Proceedings of the IRE 35, no. 8 (August 1947): 756-767. DOI: 10.1109/JRPROC.1947.234265. I found it interesting that some of the design issues for flip-flops made of vacuum tubes resemble those of designing static RAM cells in CMOS. Although ENIAC's implementation is electronic, its architecture is fundamentally that of a mechanical machine.

+

The information about ENIAC in this chapter is found in a 1947 IRE article, "Electronic Computing Circuits of the ENIAC" by Arthur W. Burks Arthur W. Burks, "Electronic Computing Circuits of the ENIAC," Proceedings of the IRE 35, no. 8 (August 1947): 756-767. DOI: 10.1109/JRPROC.1947.234265. I found it interesting that some of the design issues for flip-flops made of vacuum tubes resemble those of designing static RAM cells in CMOS. Although ENIAC’s implementation is electronic, its architecture is fundamentally that of a mechanical machine.

The first computing machines which performed discrete state computation made use of ten symbols for a digit’s potential values while mimicking how humans perform manual arithmetic. Examples include Pascal’s calculator, Babbage’s machine, the mechanical calculators that came after, Aiken’s Mark computers, and ENIAC.

@@ -3181,7 +3373,7 @@ .987 si noitaton tuohtiw [9 ,8, 7] ecneuqes eht thgir ot tfel morf gnitirW -

Following the 1,000-year-old convention, I did not reverse the order of the digits of the number. For the reversed sentence to make sense, I changed 'tfel ot thgir' (right to left) to 'left to right', which is the sort of adaptation that Middle Ages translators did not universally apply. Also, I changed the brackets so that they still enclose the sequence components; otherwise, they point outward. When reading the number, the eye first lands on a digit of unknown weight. A person must scan further right to find the one’s place, then scan back while counting the digits to make sense of the number. Reversing the string is insufficient; a person must understand the sentence and change the phrase "right to left", to "left to right".

+

Following the 1,000-year-old convention, I did not reverse the order of the digits of the number. For the reversed sentence to make sense, I changed ‘tfel ot thgir’ (right to left) to ‘left to right’, which is the sort of adaptation that Middle Ages translators did not universally apply. Also, I changed the brackets so that they still enclose the sequence components; otherwise, they point outward. When reading the number, the eye first lands on a digit of unknown weight. A person must scan further right to find the one’s place, then scan back while counting the digits to make sense of the number. Reversing the string is insufficient; a person must understand the sentence and change the phrase "right to left", to "left to right".

Now consider starting with the exact same right to left string as the early translators did, but instead of making an exception for numbers, we literally reverse the entire string:

@@ -3328,7 +3520,7 @@

If an unconventional processor loads numbers sequentially as digit streams, the load instruction must possess a mechanism to detect the end of the loaded number, or it must receive the exact length prior to execution. This structural boundary is identical to the existing mechanism for loading character strings, where systems utilize length counts or explicit end terminators.

-

For serial computation utilizing little-endian notation, the processor produces the first digit of the sum immediately after receiving the first digit of the operand. If big-endian is used, the least-significant digit arrives last, forcing the system to buffer the entire number before computing the first digit of the sum. Attempting to load from the far ends of the operands fails because the physical location of the sum's far end remains undefined until the carry propagates. Resolving big-endian serial addition requires implementing signed-digit arithmetic.

+

For serial computation utilizing little-endian notation, the processor produces the first digit of the sum immediately after receiving the first digit of the operand. If big-endian is used, the least-significant digit arrives last, forcing the system to buffer the entire number before computing the first digit of the sum. Attempting to load from the far ends of the operands fails because the physical location of the sum’s far end remains undefined until the carry propagates. Resolving big-endian serial addition requires implementing signed-digit arithmetic.

 @@ -3337,7 +3529,7 @@

Data transports between hardware nodes over bundles of wires called buses. Bus specifications explicitly define the order of bits in bytes, and for contemporary machines, bytes are strictly octets. All compute processors, channel processors, and attached devices must conform to the bus’s specifications.

-

Channel processors bridge the gap between a computer's system bus and a storage bus connected to storage devices. The storage device designers adhere strictly to the storage bus specification without evaluating the host computer architecture. The compatibility burden relies entirely on selecting a channel processor that respects the established standard. Consequently, unless a programmer explicitly designs bus standards or hardware interfaces, the physical bit order within bytes remains invisible.

+

Channel processors bridge the gap between a computer’s system bus and a storage bus connected to storage devices. The storage device designers adhere strictly to the storage bus specification without evaluating the host computer architecture. The compatibility burden relies entirely on selecting a channel processor that respects the established standard. Consequently, unless a programmer explicitly designs bus standards or hardware interfaces, the physical bit order within bytes remains invisible.

Some processors physically store bits into memory bytes in different orders. However, the data read and written into the respective memories routes through a bus, and at that boundary, the bit order aligns with the applicable bus specification.

@@ -3390,10 +3582,10 @@ - - RT + + The Realizable Machine number system -

The native Integer data type for RT utilizes a high radix online number system. This number system functions as an extension of online arithmetic. Similar to online arithmetic, it employs serial most-significant-digit-first signed-digit arithmetic. In contrast to standard signed arithmetic, the radix scales significantly higher, causing a digit to span at least a byte in length. The numbers are highly scalable, and the architecture mandates an analysis step at compile time to establish rigorous precision requirements and exact range bounds. This process is detailed in the next chapter. The architecture provides explicit conversion instructions to generate alternate number formats.

+

The native Integer data type for the Realizable Machine utilizes a high radix online number system. This number system functions as an extension of online arithmetic. Similar to online arithmetic, it employs serial most-significant-digit-first signed-digit arithmetic. In contrast to standard signed arithmetic, the radix scales significantly higher, causing a digit to span at least a byte in length. The numbers are highly scalable, and the architecture mandates an analysis step at compile time to establish rigorous precision requirements and exact range bounds. This process is detailed in the next chapter. The architecture provides explicit conversion instructions to generate alternate number formats.

 @@ -3433,7 +3625,7 @@

- For programs with large memory footprints and sparse memory access patterns, the dominant factor becomes the virtual memory subsystem. Here locality again comes into play as the translation lookaside buffer provides stateful location context, and the neighbor relationship between pages is sometimes taken into account for performance reasons. However, once a program starts performing at virtual memory page fetch times instead of cache access, or even local system memory access times, the program is said to be page thrashing, and it becomes very slow. It is also possible for small programs to have memory access patterns that alias against the memory subsystem's heuristics and lead to page thrashing. For a typical system, a programmer can write a small program to demonstrate this. + For programs with large memory footprints and sparse memory access patterns, the dominant factor becomes the virtual memory subsystem. Here locality again comes into play as the translation lookaside buffer provides stateful location context, and the neighbor relationship between pages is sometimes taken into account for performance reasons. However, once a program starts performing at virtual memory page fetch times instead of cache access, or even local system memory access times, the program is said to be page thrashing, and it becomes very slow. It is also possible for small programs to have memory access patterns that alias against the memory subsystem’s heuristics and lead to page thrashing. For a typical system, a programmer can write a small program to demonstrate this.

@@ -3470,7 +3662,7 @@ 100 ns 300 days -0042-01-09 - Nearly a year later, during the Liberators' civil war. + Nearly a year later, during the Liberators’ civil war. NVMe SSD Page Swap @@ -3508,7 +3700,7 @@ The Universal Turing Machine -

The Computer Theoretic model chapter provided symbolic definitions for the Turing Machine and the RT variation. Those definitions were written as strings of characters, which the reader scanned, and presumably understood, thus demonstrating the ability of those text strings to convey meaning. Furthermore the text explained in detail how an executor could make use of those definitions so as to perform the input string transformations. As Turing originally noted, the executor could be a person. Alternatively, as the book continued on to describe in detail, the executor could be a machine that applied the input transformations automatically.

+

The Computer Theoretic model chapter provided symbolic definitions for the Turing Machine and the Realizable variation. Those definitions were written as strings of characters, which the reader scanned, and presumably understood, thus demonstrating the ability of those text strings to convey meaning. Furthermore the text explained in detail how an executor could make use of those definitions so as to perform the input string transformations. As Turing originally noted, the executor could be a person. Alternatively, as the book continued on to describe in detail, the executor could be a machine that applied the input transformations automatically.

In his original paper Alan Turing put these things together and explained that a Universal Turing Machine could read the definition of a Turing Machine from tape, and thus be a Turing Machine executor. Hence, say, a Turing Machine reads the symbolic definition of a Turing Machine from one tape, and then automatically performs the described input string transformations on another tape. Said Universal Turing Machine would then be chameleon-like, performing the function of any other Turing Machine so described on the first tape.

@@ -3520,7 +3712,7 @@

Consider then, inserting jump table instructions to handle the next-state transitions. Then a state controller can be mechanically changed into an instruction sequence with embedded jump table instructions without having to run it and observe it. Accordingly, first examine the state diagram for the controller. Take all the sequential state runs, even those of length 1, from the controller, and list their instructions in the same sequence order. Then, after each such sequence, wherever a state has next-state transition arcs based on the value of the status register, insert a jump table instruction, so that it jumps to the appropriate instruction sequence.

-

Applying this mechanical procedure to the RT two-headed reverse string example controller results in:

+

Applying this mechanical procedure to the two-headed Realizable reverse string example controller results in:

@@ -3635,7 +3827,7 @@ Code . The same controller using test, branch and jump instructions -

This is an assembly-level code description of an instruction sequence. To get it into final form, the labels that appear as instruction arguments would be replaced with their addresses. Symbolic labels would not appear on the tape. As an alternative to using absolute branches, relative branches could be used. Performance will be greatly enhanced for a sequencer that performs relative branching if it contains an adder circuit; otherwise, the addition of offsets would be another RT Machine program invocation.

+

This is an assembly-level code description of an instruction sequence. To get it into final form, the labels that appear as instruction arguments would be replaced with their addresses. Symbolic labels would not appear on the tape. As an alternative to using absolute branches, relative branches could be used. Performance will be greatly enhanced for a sequencer that performs relative branching if it contains an adder circuit; otherwise, the addition of offsets would be another Realizable Machine program invocation.

 @@ -3646,7 +3838,7 @@ Natural Symbol -

The design for the RT Machine was given in chapter . On that machine there was a column of patch panel holes said to be symbols for gating next state transitions. The presence of a symbol could be ascertained by its effect on the machine by observing the indicator lights, or more directly if the observer is allowed and facilitated to probe the machine with a voltage meter. Alternatively, the symbols could be enumerated in the abstract, as the maker of the control panel did, when he wrote symbol names next to each of the symbol holes. These are all natural actions. When there are merely two such symbols the machine is said to be a "binary" machine. +

The design for the Realizable Machine was given in chapter . On that machine there was a column of patch panel holes said to be symbols for gating next state transitions. The presence of a symbol could be ascertained by its effect on the machine by observing the indicator lights, or more directly if the observer is allowed and facilitated to probe the machine with a voltage meter. Alternatively, the symbols could be enumerated in the abstract, as the maker of the control panel did, when he wrote symbol names next to each of the symbol holes. These are all natural actions. When there are merely two such symbols the machine is said to be a "binary" machine.

@@ -3670,7 +3862,7 @@ Naturally derived Math Symbol

- Because the RT Machine runs programs, it is possible to extend the notion of the Natural Symbol to that of the symbol of mathematics. Accordingly, the symbol of mathematics is defined as a program that produces symbol instances. A new symbol instance of the given symbol is then made, say, by calling a symbol factory's make function. All of the symbol instances made by the factory constitute the members of corresponding 'made by' equivalence class. + Because the Realizable Machine runs programs, it is possible to extend the notion of the Natural Symbol to that of the symbol of mathematics. Accordingly, the symbol of mathematics is defined as a program that produces symbol instances. A new symbol instance of the given symbol is then made, say, by calling a symbol factory’s make function. All of the symbol instances made by the factory constitute the members of corresponding ‘made by’ equivalence class.

@@ -3770,7 +3962,7 @@ Alphabet

- An alphabet is another name for 'a set of symbols'. Because it is simply a set, an infinite number of distinct alphabets can exist, and a set of symbols could be finite or infinite.

+ An alphabet is another name for ‘a set of symbols’. Because it is simply a set, an infinite number of distinct alphabets can exist, and a set of symbols could be finite or infinite.

For a Turing Machine there are a fixed number of alphabets that are all finite. The set of data symbols, control symbols, and controller state name symbols, are all finite and set in advance.

@@ -3913,11 +4105,11 @@ A Turing Machine tape cell

- A cell is the square from Alan Turing's 1936 paper Alan M. Turing, "On Computable Numbers, with an Application to the Entscheidungsproblem," Proceedings of the London Mathematical Society s2 42, no. 1 (1936): 230 265.. Mathematically, a cell is a referencable distinct identifiable set, with one to three property members depending on the type of cell. + A cell is the square from Alan Turing’s 1936 paper Alan M. Turing, "On Computable Numbers, with an Application to the Entscheidungsproblem," Proceedings of the London Mathematical Society s2 42, no. 1 (1936): 230 265.. Mathematically, a cell is a referencable distinct identifiable set, with one to three property members depending on the type of cell.

- A leftmost cell is a set of two properties named 'right neighbor' and 'data'. A rightmost cell is a set of two properties named 'left neighbor' and 'data'. An medial cell is a set holding three properties: a 'left neighbor', a 'right neighbor', and 'data'. While an island cell owns a single compulsory property, that of 'data'. + A leftmost cell is a set of two properties named ‘right neighbor’ and ‘data’. A rightmost cell is a set of two properties named ‘left neighbor’ and ‘data’. An medial cell is a set holding three properties: a ‘left neighbor’, a ‘right neighbor’, and ‘data’. While an island cell owns a single compulsory property, that of ‘data’.

@@ -3932,10 +4124,10 @@ Mathematical Tape -

The RT Machine is a natural object. We observe its mechanisms, name its parts, and then find symbol representation for them so as to facilitate introspection. In short, we build a model.

+

The Realizable Machine is a natural object. We observe its mechanisms, name its parts, and then find symbol representation for them so as to facilitate introspection. In short, we build a model.

- A tape is a set containing exactly one leftmost cell and an infinite number of medial cells. For each cell in the set, called cell A, if cell A has a right neighbor that is cell B, then cell B's left neighbor is cell A. Similarly, if cell A has a left neighbor of cell B, then cell B's right neighbor is cell A. Furthermore, any cell in the set must be reachable by traversing right neighbors starting from the leftmost cell, in a finite number of steps. + A tape is a set containing exactly one leftmost cell and an infinite number of medial cells. For each cell in the set, called cell A, if cell A has a right neighbor that is cell B, then cell B’s left neighbor is cell A. Similarly, if cell A has a left neighbor of cell B, then cell B’s right neighbor is cell A. Furthermore, any cell in the set must be reachable by traversing right neighbors starting from the leftmost cell, in a finite number of steps.

@@ -3943,7 +4135,7 @@

- In conventional computation theory, once a tape is defined, the cell neighbor properties values are fixed. Cells don't move, new cells cannot be added, and cells existing on the tape cannot be removed. This fits the definition of a space, so we can say that a Turing Machine tape has a constant, fixed, linear topology. This permanence of cells matches the reality of hardware memories. On the other hand, it does not track well with general memory containers such as linked lists where destructive operations are often permitted. + In conventional computation theory, once a tape is defined, the cell neighbor properties values are fixed. Cells don’t move, new cells cannot be added, and cells existing on the tape cannot be removed. This fits the definition of a space, so we can say that a Turing Machine tape has a constant, fixed, linear topology. This permanence of cells matches the reality of hardware memories. On the other hand, it does not track well with general memory containers such as linked lists where destructive operations are often permitted.

@@ -3955,9 +4147,9 @@

- Mathematically, a Turing Machine tape is a specialized path graph. The neighbor properties are the edges. In this special form, properties are attached to the edges. A Turing Machine has a clock based state controller. Taking a step is an atomic operation. The machine is either in state q_i or in state q_{i+1}, there is no mathematical meaning given to the concept of during a step, which is why no properties are added to the edges of the tape path graph. This is not to say that some analysis of a Turing Machine program won't take pairs of nodes from the path graph and give them meaning, thus effectively giving properties to the arcs. However, this would not be part of the machine definition, such a program must go through the additional effort of making node pairs, because the machine itself does not provide the program with a feature for attaching properties directly to the neighbor property itself.

+ Mathematically, a Turing Machine tape is a specialized path graph. The neighbor properties are the edges. In this special form, properties are attached to the edges. A Turing Machine has a clock based state controller. Taking a step is an atomic operation. The machine is either in state q_i or in state q_{i+1}, there is no mathematical meaning given to the concept of during a step, which is why no properties are added to the edges of the tape path graph. This is not to say that some analysis of a Turing Machine program won’t take pairs of nodes from the path graph and give them meaning, thus effectively giving properties to the arcs. However, this would not be part of the machine definition, such a program must go through the additional effort of making node pairs, because the machine itself does not provide the program with a feature for attaching properties directly to the neighbor property itself.

-

An example of a non-Turing tape like model is the Emacs vertical line cursor model, where a cursor is said to be between characters. An ASCII file offers no such feature as 'in between' characters. Like a Turing Machine tape, a medial character in an ASCII file has a left neighbor and a right neighbor character. Any attempt to represent an in between cursor within the file itself would require inserting more characters into the file under the same model of every medial character having a left and a right character. Consequently, though emacs presents a model where cursor is located in between two characters, this model is only due to the interpretation of the functions' actual effects presented to users in the documentation. For example, instead of saying a cursor is located upon a character, and that inserting a character inserts the character to the right of the given character, the manual gives the description that the new character is inserted at the cursor location, where said cursor is in between the given character and its right neighbor. Thus the documentation presents the user with one model, which by necessity of using standard library calls to work with files, is built upon another model.

+

An example of a non-Turing tape like model is the Emacs vertical line cursor model, where a cursor is said to be between characters. An ASCII file offers no such feature as ‘in between’ characters. Like a Turing Machine tape, a medial character in an ASCII file has a left neighbor and a right neighbor character. Any attempt to represent an in between cursor within the file itself would require inserting more characters into the file under the same model of every medial character having a left and a right character. Consequently, though emacs presents a model where cursor is located in between two characters, this model is only due to the interpretation of the functions’ actual effects presented to users in the documentation. For example, instead of saying a cursor is located upon a character, and that inserting a character inserts the character to the right of the given character, the manual gives the description that the new character is inserted at the cursor location, where said cursor is in between the given character and its right neighbor. Thus the documentation presents the user with one model, which by necessity of using standard library calls to work with files, is built upon another model.

 @@ -3984,7 +4176,7 @@

- This analytical perspective has been present from the very beginning of the field. Alan Turing's formulation of the halting problem relies explicitly on one machine examining another. To properly evaluate the limits of this analytical capacity, the examining machine must necessarily be given definitions of programs that loop infinitely. Consequently, the foundational proofs of computer science formally establish non computational programs as legitimate objects of analytical study. + This analytical perspective has been present from the very beginning of the field. Alan Turing’s formulation of the halting problem relies explicitly on one machine examining another. To properly evaluate the limits of this analytical capacity, the examining machine must necessarily be given definitions of programs that loop infinitely. Consequently, the foundational proofs of computer science formally establish non computational programs as legitimate objects of analytical study.

@@ -4001,11 +4193,11 @@ second-order Arithmetic

- In their 1990 paper, "Exact Real Arithmetic: Formulating Real Numbers as Functions," Hans-Juergen Boehm and Robert Cartwright presented a system using Turing Machines to represent 'constructive real numbers', or exact real arithmetic. This directly implements a core concept from Alan Turing's 1936 paper of the "computable number." Hans-J. Boehm and Robert Cartwright, "Exact Real Arithmetic: Formulating Real Numbers as Functions," in D. A. Turner, ed., Research Topics in Functional Programming (Reading: Addison-Wesley, 1990), 43–64. The original presentation is Hans-J. Boehm, Robert Cartwright, Mark Riggle, and Michael J. O'Donnell, "Exact Real Arithmetic: A Case Study in Higher Order Programming," in Proceedings of the 1986 ACM Conference on LISP and Functional Programming (New York: ACM Press, 1986), 162–173. Boehm translated this computation theory into a practical software architecture. Instead of storing a real number as a fixed-width floating-point approximation, Boehm's system represents a real number as a lazy evaluation function. + In their 1990 paper, "Exact Real Arithmetic: Formulating Real Numbers as Functions," Hans-Juergen Boehm and Robert Cartwright presented a system using Turing Machines to represent ‘constructive real numbers’, or exact real arithmetic. This directly implements a core concept from Alan Turing’s 1936 paper of the "computable number." Hans-J. Boehm and Robert Cartwright, "Exact Real Arithmetic: Formulating Real Numbers as Functions," in D. A. Turner, ed., Research Topics in Functional Programming (Reading: Addison-Wesley, 1990), 43–64. The original presentation is Hans-J. Boehm, Robert Cartwright, Mark Riggle, and Michael J. O’Donnell, "Exact Real Arithmetic: A Case Study in Higher Order Programming," in Proceedings of the 1986 ACM Conference on LISP and Functional Programming (New York: ACM Press, 1986), 162–173. Boehm translated this computation theory into a practical software architecture. Instead of storing a real number as a fixed-width floating-point approximation, Boehm’s system represents a real number as a lazy evaluation function.

- Let us employ Boehm and Cartwright's constructive real numbers, though in second-order arithmetic rather than a lazy evaluation system. Because we are doing second-order arithmetic, our programs need not be in the form of generators that yield a new value when called, but can be arbitrary programs containing complex control logic. If run, they evaluate physical conditions and produce results. As we are in the second-order, however, we will not run them, but rather analyze and manipulate their structural logic. + Let us employ Boehm and Cartwright’s constructive real numbers, though in second-order arithmetic rather than a lazy evaluation system. Because we are doing second-order arithmetic, our programs need not be in the form of generators that yield a new value when called, but can be arbitrary programs containing complex control logic. If run, they evaluate physical conditions and produce results. As we are in the second-order, however, we will not run them, but rather analyze and manipulate their structural logic.

@@ -4013,7 +4205,7 @@

- Suppose our goal is to subtract 3 from 5 in the second-order. Consider a Turing Machine representation named five that outputs the number 5, i.e., it prints to the tape, 'sssss', using unary notation. Let us assume that the tape is initially empty and that the empty symbol '□' terminates the string. To preserve the code for second-order analysis, we define the program's Abstract Syntax Tree (AST) as a quoted progn block. This block can contain any native Lisp control structures, though for this generator it is a simple sequence: + Suppose our goal is to subtract 3 from 5 in the second-order. Consider a Turing Machine representation named five that outputs the number 5, i.e., it prints to the tape, ‘sssss’, using unary notation. Let us assume that the tape is initially empty and that the empty symbol ‘□’ terminates the string. To preserve the code for second-order analysis, we define the program’s Abstract Syntax Tree (AST) as a quoted progn block. This block can contain any native Lisp control structures, though for this generator it is a simple sequence:

@@ -4051,7 +4243,7 @@

- For the second-order difference operation, we compose the ASTs to create a new program. Here we extract the body of the second operand's AST and substitute every (write s) followed by a (step) instruction with an inverted pair: a (left) instruction followed by writing the empty symbol (write □). + For the second-order difference operation, we compose the ASTs to create a new program. Here we extract the body of the second operand’s AST and substitute every (write s) followed by a (step) instruction with an inverted pair: a (left) instruction followed by writing the empty symbol (write □).

@@ -4116,7 +4308,7 @@

- I called this 'primitive' because it is missing the simplification. This sort of simplification is also known as compiler code optimization. Optimizations can be operation specific, or general in nature. Here we will send the primitive composition's body to an analyzer which scans the steps and removes the redundant operations. + I called this ‘primitive’ because it is missing the simplification. This sort of simplification is also known as compiler code optimization. Optimizations can be operation specific, or general in nature. Here we will send the primitive composition’s body to an analyzer which scans the steps and removes the redundant operations.

@@ -4124,7 +4316,7 @@

- For a program containing branched control logic, the simplifier would require a deep recursive walk of the AST to ensure operations aren't annihilated across conditional boundaries. For our explicit Peano Number generators, a linear scan of the progn body suffices. It calls itself recursively until the scan fails to find any further reductions, returning its optimized AST. + For a program containing branched control logic, the simplifier would require a deep recursive walk of the AST to ensure operations aren’t annihilated across conditional boundaries. For our explicit Peano Number generators, a linear scan of the progn body suffices. It calls itself recursively until the scan fails to find any further reductions, returning its optimized AST.

@@ -4263,7 +4455,7 @@ δ_i = f(t_{i+1}) - f(t_i). While the backwards first difference is δ_i = f(t_i) - f(t_{i-1}). - If 'forward' or 'backwards' is not specified, then the difference is taken to be 'forward'. Hence, extending a function is identical to adding the first difference. Thus a sequence of function values implies a sequence of first differences. + If ‘forward’ or ‘backwards’ is not specified, then the difference is taken to be ‘forward’. Hence, extending a function is identical to adding the first difference. Thus a sequence of function values implies a sequence of first differences.

@@ -4397,7 +4589,7 @@

- The exact algebraic composition of this mechanical expansion is formalized by Newton's calculus of finite differences Isaac Newton formalized this interpolation method in 1675, later published in his Methodus Differentialis (1711). For a comprehensive foundational treatment, see George Boole, A Treatise on the Calculus of Finite Differences (Cambridge: Macmillan and Co., 1860), Chapter II.. Instead of executing the machine incrementally, a person can calculate the function value at call \omega directly as a linear combination of the initial tape components using Newton's forward difference formula: + The exact algebraic composition of this mechanical expansion is formalized by Newton’s calculus of finite differences Isaac Newton formalized this interpolation method in 1675, later published in his Methodus Differentialis (1711). For a comprehensive foundational treatment, see George Boole, A Treatise on the Calculus of Finite Differences (Cambridge: Macmillan and Co., 1860), Chapter II.. Instead of executing the machine incrementally, a person can calculate the function value at call \omega directly as a linear combination of the initial tape components using Newton’s forward difference formula:

@@ -4405,7 +4597,7 @@ D_{\omega, 0} = \sum_{j=0}^{\omega} \binom{\omega}{j} D_{0, j} - eq: Newton's forward difference formula for the value at call ω + eq: Newton’s forward difference formula for the value at call ω

@@ -4447,7 +4639,7 @@

- Consider an extent of \omega = 0, forming a polynomial of degree 0, f(t) = a_0. Evaluating at t = 0 yields f(0) = a_0. Because the machine's initial value is D_{0,0}, a person establishes a_0 = D_{0,0}. + Consider an extent of \omega = 0, forming a polynomial of degree 0, f(t) = a_0. Evaluating at t = 0 yields f(0) = a_0. Because the machine’s initial value is D_{0,0}, a person establishes a_0 = D_{0,0}.

@@ -4555,7 +4747,7 @@

- This mechanical recovery of standard polynomial constants is completely analogous to Newton's interpolation formula Ibid.. Instead of resolving the standard constants a_i through an upper triangular matrix, a person can construct the polynomial directly by treating the initial tape components as the exact coefficients for a basis of binomial terms: + This mechanical recovery of standard polynomial constants is completely analogous to Newton’s interpolation formula Ibid.. Instead of resolving the standard constants a_i through an upper triangular matrix, a person can construct the polynomial directly by treating the initial tape components as the exact coefficients for a basis of binomial terms:

@@ -4695,7 +4887,7 @@

- Suppose we move to a second-order evaluation, and for places the computation does not work, we return the machine that isn't evaluating. That is similar to what we did to get (divide 0 0), but the zeros in that expression came from a prior step first-order evaluation. Let us instead give the quotient composer two machines to compose, and have it return a value when reduction to the first-order is possible, and return the full problem as posed to it when it can not be reduced. + Suppose we move to a second-order evaluation, and for places the computation does not work, we return the machine that isn’t evaluating. That is similar to what we did to get (divide 0 0), but the zeros in that expression came from a prior step first-order evaluation. Let us instead give the quotient composer two machines to compose, and have it return a value when reduction to the first-order is possible, and return the full problem as posed to it when it can not be reduced.

@@ -4776,14 +4968,14 @@

- The quotient machine, (divide f(5) g(5)), is no more representative of the value we expected here, than was (divide 0 0). However, as we have the whole program, we could pass it to a L'Hôpital evaluator and get a value back. But I am wondering if we can't design function extension form that extends to the value and beyond without any hiccups, as most apparently there is a value to extend to. + The quotient machine, (divide f(5) g(5)), is no more representative of the value we expected here, than was (divide 0 0). However, as we have the whole program, we could pass it to a L’Hôpital evaluator and get a value back. But I am wondering if we can’t design function extension form that extends to the value and beyond without any hiccups, as most apparently there is a value to extend to.

I included the first difference along with the evaluation results in Table . When building a first difference table an interesting thing happens at the singularity, the first difference and the function itself coincide, note calls 4 and 5. This makes sense because δ f(4) = f(5) - f(4), which becomes δ f(4) = 0 - f(4), and the same happens to g in the denominator, so the signs cancel. Note also δ f(5) = f(6) - f(5), which becomes δ f(5) = f(6) - 0.

-

Ostensibly it looks like we have happened upon a discrete version of L'Hôpital's rule, that we merely use the first difference quotient instead of the function quotient at the singularity. But alas, the coincidence occurs at h(4) rather than at h(5). For the second coincidence, we find h(6) as the 5th first finite difference. In neither case did we learn anything about the value of h(5). +

Ostensibly it looks like we have happened upon a discrete version of L’Hôpital’s rule, that we merely use the first difference quotient instead of the function quotient at the singularity. But alas, the coincidence occurs at h(4) rather than at h(5). For the second coincidence, we find h(6) as the 5th first finite difference. In neither case did we learn anything about the value of h(5).

@@ -4821,11 +5013,11 @@

- As the second-order evaluation returned (divide f(k)|k=5 g(k)|k=5), we can use L'Hôpital evaluator to resolve this indeterminate form D_{0,5}. Note this is during the creation of the D_0 vector, so it is part of the simplification step, rather than being part of the evaluation of the quotient. The result will thus be used in producing an extension machine that extends through the singularity, so the user will not see any exceptional higher-order values while extending h(t). + As the second-order evaluation returned (divide f(k)|k=5 g(k)|k=5), we can use L’Hôpital evaluator to resolve this indeterminate form D_{0,5}. Note this is during the creation of the D_0 vector, so it is part of the simplification step, rather than being part of the evaluation of the quotient. The result will thus be used in producing an extension machine that extends through the singularity, so the user will not see any exceptional higher-order values while extending h(t).

- The L'Hôpital evaluator will discover that a transcendental constant is required. It will be returned as a higher-order object, a machine definition, because the value can not be written to the tape. + The L’Hôpital evaluator will discover that a transcendental constant is required. It will be returned as a higher-order object, a machine definition, because the value can not be written to the tape.

@@ -4861,7 +5053,7 @@

- So the value of h(5) is transcendental, but all the machinery we were using, up until using L'Hôpital evaluator, was closed over the Rational field. That is why no matter how we tried to organize the difference tables, there was no way to find h(5), or to find a D table for extending through it. This situation occurred due to introducing the 2^t function. Had we remained with finite length D tables, i.e. finite degree polynomials, we could have used deconvolution of the D_0 vectors, or synthetic division on the polynomials, to create the quotient. + So the value of h(5) is transcendental, but all the machinery we were using, up until using L’Hôpital evaluator, was closed over the Rational field. That is why no matter how we tried to organize the difference tables, there was no way to find h(5), or to find a D table for extending through it. This situation occurred due to introducing the 2^t function. Had we remained with finite length D tables, i.e. finite degree polynomials, we could have used deconvolution of the D_0 vectors, or synthetic division on the polynomials, to create the quotient.

@@ -4949,31 +5141,31 @@ Code . A simple cubic function -

It is said to be a continuous function over the real field. However, from a computational perspective, it is a string of 8 discrete symbols. A person might ask then, "If the function is not continuous, then how can a person take a derivative?" Often people view a derivative as a tangent line to a curve drawn on a plot. The computational analyst's answer is to use a machine that manipulates the symbols; it will move the 3 down in front of the t, decrement the power, and write 2. Those are all discrete operations.

+

It is said to be a continuous function over the real field. However, from a computational perspective, it is a string of 8 discrete symbols. A person might ask then, "If the function is not continuous, then how can a person take a derivative?" Often people view a derivative as a tangent line to a curve drawn on a plot. The computational analyst’s answer is to use a machine that manipulates the symbols; it will move the 3 down in front of the t, decrement the power, and write 2. Those are all discrete operations.

The point of this chapter is to state that computational analysis exists, rather than to present a comprehensive computational analysis system. If a person is looking for practical systems for extending functions that expand precision, or for second-order computation, there are options.

-

The oldest of the options carries the error inside the value rather than reporting it alongside. In interval arithmetic a quantity is a pair of bounds, and every operation returns an interval guaranteed to contain the true result, so a computation ends holding an answer together with a proof of that answer's accuracy. Ramon Moore conceived of the method in 1958, set out its principles in his 1962 Stanford dissertation, and gave it its standard treatment in Interval Analysis.R. E. Moore, Interval Arithmetic and Automatic Error Analysis in Digital Computing, Ph.D. dissertation, Department of Mathematics, Stanford University, November 1962, issued as Applied Mathematics and Statistics Laboratories Technical Report No. 25. R. E. Moore, Interval Analysis (Englewood Cliffs: Prentice-Hall, 1966); and Methods and Applications of Interval Analysis (Philadelphia: SIAM, 1979). The method now has a standard of its own, IEEE Std 1788-2015. +

The oldest of the options carries the error inside the value rather than reporting it alongside. In interval arithmetic a quantity is a pair of bounds, and every operation returns an interval guaranteed to contain the true result, so a computation ends holding an answer together with a proof of that answer’s accuracy. Ramon Moore conceived of the method in 1958, set out its principles in his 1962 Stanford dissertation, and gave it its standard treatment in Interval Analysis.R. E. Moore, Interval Arithmetic and Automatic Error Analysis in Digital Computing, Ph.D. dissertation, Department of Mathematics, Stanford University, November 1962, issued as Applied Mathematics and Statistics Laboratories Technical Report No. 25. R. E. Moore, Interval Analysis (Englewood Cliffs: Prentice-Hall, 1966); and Methods and Applications of Interval Analysis (Philadelphia: SIAM, 1979). The method now has a standard of its own, IEEE Std 1788-2015.

-

Kulisch and Miranker's screen is what joins interval arithmetic to the formalization of computer arithmetic. The values a machine can represent form a screen lying over the real number space, and a rounding carries the reals onto it. An operation is then defined first over the powerset of the reals, where it is exact, and that exact result is rounded onto the screen, or onto the intervals over the screen, by taking the least interval which encloses it. Machine arithmetic becomes the image of exact arithmetic under a rounding, and the interval is what the rounding leaves behind, which is to say it is the error made part of the value. Kulisch carried the same principle to the exact dot product, where a long accumulator holds a sum of products with no intermediate rounding at all, and to the XSC languages, which put both the intervals and the accumulator in reach of a programmer.Ulrich W. Kulisch and Willard L. Miranker, Computer Arithmetic in Theory and Practice (New York: Academic Press, 1981). For the extension to product spaces see the same authors' "Arithmetic Operations in Interval Spaces," in G. Alefeld and R. D. Grigorieff, eds., Fundamentals of Numerical Computation, Computing Supplementum vol. 2 (Vienna: Springer, 1980), 51–67. +

Kulisch and Miranker’s screen is what joins interval arithmetic to the formalization of computer arithmetic. The values a machine can represent form a screen lying over the real number space, and a rounding carries the reals onto it. An operation is then defined first over the powerset of the reals, where it is exact, and that exact result is rounded onto the screen, or onto the intervals over the screen, by taking the least interval which encloses it. Machine arithmetic becomes the image of exact arithmetic under a rounding, and the interval is what the rounding leaves behind, which is to say it is the error made part of the value. Kulisch carried the same principle to the exact dot product, where a long accumulator holds a sum of products with no intermediate rounding at all, and to the XSC languages, which put both the intervals and the accumulator in reach of a programmer.Ulrich W. Kulisch and Willard L. Miranker, Computer Arithmetic in Theory and Practice (New York: Academic Press, 1981). For the extension to product spaces see the same authors’ "Arithmetic Operations in Interval Spaces," in G. Alefeld and R. D. Grigorieff, eds., Fundamentals of Numerical Computation, Computing Supplementum vol. 2 (Vienna: Springer, 1980), 51–67.

-

What interval arithmetic does not do is keep its bounds tight. An interval records the range of a quantity but forgets the quantity's identity, so the two occurrences of x in x - x are treated as though they were independent, and the result is an interval about zero rather than zero. Bounds therefore widen through a long computation whether or not the underlying quantities are correlated, until what they report is rigorous and useless together. This dependency problem is why interval arithmetic, for all that it proves what it claims, did not displace the alternatives, and it is why the systems described next pursue precision rather than bounds.

+

What interval arithmetic does not do is keep its bounds tight. An interval records the range of a quantity but forgets the quantity’s identity, so the two occurrences of x in x - x are treated as though they were independent, and the result is an interval about zero rather than zero. Bounds therefore widen through a long computation whether or not the underlying quantities are correlated, until what they report is rigorous and useless together. This dependency problem is why interval arithmetic, for all that it proves what it claims, did not displace the alternatives, and it is why the systems described next pursue precision rather than bounds.

-

Hans J. Boehm and Robert Cartwright's work of expanding the precision of a computation on demand continued to mature. Boehm, while working at Google, authored a Java constructive reals library that powers the bundled Android calculator. When a person types in an expression, the system builds an Abstract Syntax Tree (AST) of the expression. The display interface determines how many digits fit on the screen and demands exactly that much precision from the root of the tree. The root function then recursively demands progressively higher precision from its operand functions until it achieves the strict error bounds required to guarantee every digit displayed on the screen is mathematically correct Hans J. Boehm, "Small data computing: Correct calculator arithmetic," Google Research, 2015. This library uses constructive real arithmetic to guarantee fully accurate results through demand driven evaluation.. +

Hans J. Boehm and Robert Cartwright’s work of expanding the precision of a computation on demand continued to mature. Boehm, while working at Google, authored a Java constructive reals library that powers the bundled Android calculator. When a person types in an expression, the system builds an Abstract Syntax Tree (AST) of the expression. The display interface determines how many digits fit on the screen and demands exactly that much precision from the root of the tree. The root function then recursively demands progressively higher precision from its operand functions until it achieves the strict error bounds required to guarantee every digit displayed on the screen is mathematically correct Hans J. Boehm, "Small data computing: Correct calculator arithmetic," Google Research, 2015. This library uses constructive real arithmetic to guarantee fully accurate results through demand driven evaluation..

-

Wolfram Mathematica utilizes a true second-order computation system by keeping expressions in their exact symbolic form for as long as possible. An expression like Sqrt[2] + Pi remains an AST of symbols. It then relies on an extensive library of algebraic rewrite rules to simplify the tree. However, unlike Boehm and Cartwright's method of pulling precision through back propagating extension calls, when a numeric result is required, Mathematica runs the computation forward using significance arithmetic. Should there be insufficient precision at the result, it employs an adaptive retry loop Fredrik Johansson, "The significance of arithmetic," 2008. Mathematica implements significance arithmetic by approximating numbers as a floating-point value with an attached error estimate, allowing dynamic adaptation and iteration if precision falls below the required threshold. See also: Wolfram Language Documentation, "Arbitrary Precision Numbers" (https://reference.wolfram.com/language/tutorial/ArbitraryPrecisionNumbers.html).. +

Wolfram Mathematica utilizes a true second-order computation system by keeping expressions in their exact symbolic form for as long as possible. An expression like Sqrt[2] + Pi remains an AST of symbols. It then relies on an extensive library of algebraic rewrite rules to simplify the tree. However, unlike Boehm and Cartwright’s method of pulling precision through back propagating extension calls, when a numeric result is required, Mathematica runs the computation forward using significance arithmetic. Should there be insufficient precision at the result, it employs an adaptive retry loop Fredrik Johansson, "The significance of arithmetic," 2008. Mathematica implements significance arithmetic by approximating numbers as a floating-point value with an attached error estimate, allowing dynamic adaptation and iteration if precision falls below the required threshold. See also: Wolfram Language Documentation, "Arbitrary Precision Numbers" (https://reference.wolfram.com/language/tutorial/ArbitraryPrecisionNumbers.html)..

The approach I used differs from both of these approaches. Like them, the first step is to build the AST, but then the tree is evaluated in two phases. The first phase analyzes the tree, while creating error propagation expressions either per computation step or for atomic computation blocks. These take the form of symbolic expressions with error \epsilon in and error δ out, over the domain of the input. Then instead of back propagation of precision requests at run time, there is a back propagation of precision requirements at analysis time. This approach was used on the AMD K5 processor transcendental microcode to guarantee that computations yielded specified accuracy. The method is also well suited for setting the bus widths for application specific processors, because the analysis step can occur once at design time. Thomas Walker Lynch, A. Ahmed, M. Schulte, T. Callaway, and R. Tisdale, "The K5 Transcendental Functions," Proceedings of the 12th IEEE Symposium on Computer Arithmetic, 1995. DOI: 10.1109/ARITH.1995.465368. Thomas Walker Lynch, "Numerical Analysis of Computer Approximations," September 2018. DOI: 10.13140/RG.2.2.10906.49601. License CC BY 4.0. https://www.researchgate.net/publication/329402329_Numerical_Analysis_of_Computer_Approximations.

-

None of these computation systems, that of Boehm and Cartwright, Mathematica, nor my error analysis approach, makes use of the IEEE 754 standard floating-point arithmetic. Rather they all require the use of variable precision. In the case of Boehm and Cartwright's this occurs through the serialization implied through lazy calls for more precision. In the case of Mathematica it is explicit in the significance arithmetic. In my error analysis approach, the means for variable precision was the High Radix Online Arithmetic Thomas W. Lynch, "High Radix On Line Arithmetic for Credible and Accurate Computing," Real Numbers and Computers, École des Mines de Saint Étienne, France, 1995, pp. 78 89. Thomas W. Lynch and Michael J. Schulte, "Software for High Radix On Line Arithmetic," Reliable Computing, vol. 2, no. 2, 1996, pp. 133 138. DOI: 10.1007/BF02425915.. +

None of these computation systems, that of Boehm and Cartwright, Mathematica, nor my error analysis approach, makes use of the IEEE 754 standard floating-point arithmetic. Rather they all require the use of variable precision. In the case of Boehm and Cartwright’s this occurs through the serialization implied through lazy calls for more precision. In the case of Mathematica it is explicit in the significance arithmetic. In my error analysis approach, the means for variable precision was the High Radix Online Arithmetic Thomas W. Lynch, "High Radix On Line Arithmetic for Credible and Accurate Computing," Real Numbers and Computers, École des Mines de Saint Étienne, France, 1995, pp. 78 89. Thomas W. Lynch and Michael J. Schulte, "Software for High Radix On Line Arithmetic," Reliable Computing, vol. 2, no. 2, 1996, pp. 133 138. DOI: 10.1007/BF02425915..

-

Hence what would be useful for a replacement of the IEEE 754, would be a variable precision number standard. John Gustafson's earlier unum proposal was of that kind, carrying an operand of variable width. His later and more widely taken up Posit representation moved away from variable width operands in favour of a fixed width whose internal field boundaries move, so that accuracy is greatest for magnitudes near one and tapers toward the extremes of the range. A Posit is therefore an improvement upon the IEEE 754 within a fixed width, rather than the variable precision standard called for here John L. Gustafson, The End of Error: Unum Computing (Boca Raton: Chapman and Hall/CRC, 2015). John L. Gustafson and Isaac T. Yonemoto, "Beating Floating Point at its Own Game: Posit Arithmetic," Supercomputing Frontiers and Innovations 4, no. 2 (2017): 71–86. John L. Gustafson, Every Bit Counts: Posit Computing (Boca Raton: Chapman and Hall/CRC, 2024). The 2017 paper states plainly that, unlike the earlier unum forms, Posits require neither interval arithmetic nor variable size operands.. +

Hence what would be useful for a replacement of the IEEE 754, would be a variable precision number standard. John Gustafson’s earlier unum proposal was of that kind, carrying an operand of variable width. His later and more widely taken up Posit representation moved away from variable width operands in favour of a fixed width whose internal field boundaries move, so that accuracy is greatest for magnitudes near one and tapers toward the extremes of the range. A Posit is therefore an improvement upon the IEEE 754 within a fixed width, rather than the variable precision standard called for here John L. Gustafson, The End of Error: Unum Computing (Boca Raton: Chapman and Hall/CRC, 2015). John L. Gustafson and Isaac T. Yonemoto, "Beating Floating Point at its Own Game: Posit Arithmetic," Supercomputing Frontiers and Innovations 4, no. 2 (2017): 71–86. John L. Gustafson, Every Bit Counts: Posit Computing (Boca Raton: Chapman and Hall/CRC, 2024). The 2017 paper states plainly that, unlike the earlier unum forms, Posits require neither interval arithmetic nor variable size operands..

This section reviewed finite difference methods as a means to implement function extensions. A person should notice that finite differences are one among many methods for extending functions, and they are not always the best choice. In modern computing it is not common to implement function extensions. Rather it is more common to increment a value across a domain, while completely repeating the evaluation for each new domain value, for example when computing values to place on a plot. However, independent of their importance in applied computing, function extension and difference methods are of theoretical importance for analysis, particularly when applied to nonstandard analysis. These concepts speak directly to the thesis of computational naturalism—deriving mathematics from computational logic—rather than merely proposing practical methods for implementing software. @@ -5022,7 +5214,7 @@ 100 ns 300 days -0042-01-09 - Nearly a year later, during the Liberators' civil war. + Nearly a year later, during the Liberators’ civil war. NVMe SSD Page Swap @@ -5058,7 +5250,7 @@

- Professor Gonzalez once observed that the primary purpose of computer architecture is to execute the customer's programs as rapidly as possible. IBM later amended this philosophy to add 'for what they paid for,' famously designing a computer model where performance was artificially restricted unless the customer paid to have a physical hardware jumper removed. Within this prevailing design philosophy, the graceful handling of computational end cases, where most formal theoretical questions arise, is deemed secondary because such cases do not occur frequently in the critical execution path. + Professor Gonzalez once observed that the primary purpose of computer architecture is to execute the customer’s programs as rapidly as possible. IBM later amended this philosophy to add ‘for what they paid for,’ famously designing a computer model where performance was artificially restricted unless the customer paid to have a physical hardware jumper removed. Within this prevailing design philosophy, the graceful handling of computational end cases, where most formal theoretical questions arise, is deemed secondary because such cases do not occur frequently in the critical execution path.

@@ -5097,7 +5289,7 @@

- Logic is then defined on top of relay switch logic, as Shannon and others have already done. A machine that requires all inputs to be the '1' symbol to produce a '1' symbol output is a conjunction machine, and so forth. + Logic is then defined on top of relay switch logic, as Shannon and others have already done. A machine that requires all inputs to be the ‘1’ symbol to produce a ‘1’ symbol output is a conjunction machine, and so forth.

@@ -5113,15 +5305,15 @@

- Frege's set theory is then the analysis of a logic program against an enumeration of inputs to choose if a proposed symbol is in a set. Perhaps executing such a program is left to first-order analysis, or perhaps execution is not practical, and evaluation is left to second-order analysis. + Frege’s set theory is then the analysis of a logic program against an enumeration of inputs to choose if a proposed symbol is in a set. Perhaps executing such a program is left to first-order analysis, or perhaps execution is not practical, and evaluation is left to second-order analysis.

- Russell's Paradox will then be expressed as a Turing Machine that can be analyzed in the second-order, but cannot be analyzed in the first-order. That is, the paradox exists merely in the first-order as it will never halt when run. However, it is not a paradox in the second-order. It is, of course, through second-order analysis that we are able to describe why Russell's Set description does not resolve in the first-order. + Russell’s Paradox will then be expressed as a Turing Machine that can be analyzed in the second-order, but cannot be analyzed in the first-order. That is, the paradox exists merely in the first-order as it will never halt when run. However, it is not a paradox in the second-order. It is, of course, through second-order analysis that we are able to describe why Russell’s Set description does not resolve in the first-order.

- With the language of Computational Naturalism it is possible to restate every statement ever made by any mathematician; we might say that the mere fact that a mathematician was able to state something qualifies its membership into Zermelo's S set. + With the language of Computational Naturalism it is possible to restate every statement ever made by any mathematician; we might say that the mere fact that a mathematician was able to state something qualifies its membership into Zermelo’s S set.

@@ -5129,11 +5321,11 @@

- This brings us back to the reference from the Zermelo discussion in the introduction. Specifically, the question posed is if our finding through second-order analysis that Russell's paradoxical set formulation will not run in the first-order and define a set, does this mean that we are merely using S to state that we are tossing out sets that cannot be defined? The answer is it is not quite this simple. Had Frege said, 'we merely dismiss such sets', we would not have the formalization for the second-order analysis. Throwing out Russell's paradox simply because we ran it and it never halted is not a practical approach. Instead, we omit it specifically from S because analyzing R reveals that it fails to define a first-order halting machine, and S, by definition, only holds first-order halting machines. Zermelo's language is precise and formal. However, it is this bothersome nuance, now articulated here, which caused us to scratch our heads when reading Zermelo the first time. + This brings us back to the reference from the Zermelo discussion in the introduction. Specifically, the question posed is if our finding through second-order analysis that Russell’s paradoxical set formulation will not run in the first-order and define a set, does this mean that we are merely using S to state that we are tossing out sets that cannot be defined? The answer is it is not quite this simple. Had Frege said, ‘we merely dismiss such sets’, we would not have the formalization for the second-order analysis. Throwing out Russell’s paradox simply because we ran it and it never halted is not a practical approach. Instead, we omit it specifically from S because analyzing R reveals that it fails to define a first-order halting machine, and S, by definition, only holds first-order halting machines. Zermelo’s language is precise and formal. However, it is this bothersome nuance, now articulated here, which caused us to scratch our heads when reading Zermelo the first time.

- Russell's set formulation, R, can be analyzed to determine that it would not halt if it were run. We still keep it as a second-order object, and have even given it a name, R. However, this begs the question, are there machines that cannot even be analyzed in the second-order to ascertain if they would return a result in first-order analysis, i.e., when they are run? If Gödel has a say here, a person would wager that such machines exist. But then, is there an option for analysis in the third-order? Is Turing's halting proof a third-order analysis as it reasons about running the second-order analyzer? Or is it merely a recursive application of second-order analysis? + Russell’s set formulation, R, can be analyzed to determine that it would not halt if it were run. We still keep it as a second-order object, and have even given it a name, R. However, this begs the question, are there machines that cannot even be analyzed in the second-order to ascertain if they would return a result in first-order analysis, i.e., when they are run? If Gödel has a say here, a person would wager that such machines exist. But then, is there an option for analysis in the third-order? Is Turing’s halting proof a third-order analysis as it reasons about running the second-order analyzer? Or is it merely a recursive application of second-order analysis?

@@ -5241,7 +5433,7 @@ diff --git a/document/book/TM-2026_parked.html b/document/book/TM-2026_parked.html deleted file mode 100644 index 62d4e89..0000000 --- a/document/book/TM-2026_parked.html +++ /dev/null @@ -1,102 +0,0 @@ - - more appendix mateial ... - -

Multiplication of synchronized difference vectors

- -

- Because the forward difference operator is linear, adding or subtracting two polynomials is achieved by the elementwise addition or subtraction of their initial difference vectors. Multiplication, however, requires a discrete convolution of the two vectors. -

- -

- Suppose a programmer has two initial difference vectors, A_0 representing function f(t) with an extent of \omega_a, and B_0 representing function g(t) with an extent of \omega_b. The goal is to compute the initial difference vector C_0 for the product function h(t) = f(t)g(t). -

- -

- Multiplying two polynomials of degrees \omega_a and \omega_b yields a polynomial of degree \omega_a + \omega_b. Therefore, the resulting vector C_0 will strictly have an extent of \omega_c = \omega_a + \omega_b, requiring a tape component count of \omega_a + \omega_b + 1. -

- -

- To determine the components of C_0 directly from A_0 and B_0 without evaluating the functions, we rely on the multiplication of their basis elements. In the calculus of finite differences, polynomials are expanded using binomial coefficients. The product of two binomial coefficients expands into a linear combination of higher binomial coefficients according to a known combinatorial identity: -

- -

- \binom{t}{i} \binom{t}{j} = \sum_{k=\max(i,j)}^{i+j} \binom{k}{i} \binom{i}{k-j} \binom{t}{k} -

- -

- By applying this identity across the summations of both input functions, the component k of the resulting vector C_0 can be computed algebraically. Each component C_{0, k} is the sum of the cross products of the input components, weighted by combinations of their indices: -

- -

- C_{0, k} = \sum_{i=0}^{\omega_a} \sum_{j=0}^{\omega_b} A_{0, i} B_{0, j} \binom{k}{i} \binom{i}{k-j} -

- -

- Thus, while a Turing Machine extending the function only requires a simple accumulator, a machine tasked with multiplying two initial tapes must perform a combinatorial cross multiplication to generate the expanded tape before the extension sequence can begin. -

- -

Division and the Reciprocal Difference Vector

- -

- If the multiplication of two polynomials in the finite difference domain is a discrete convolution, then division is a discrete deconvolution. By finding the reciprocal of a difference vector, a programmer can perform division using the same combinatorial architecture. -

- -

- Let A_0 be the initial difference vector for a polynomial f(t). We seek the reciprocal difference vector C_0, which represents the function h(t) = 1/f(t). Because the reciprocal of a polynomial is a rational function, its forward differences will never reduce to zero. Thus, C_0 is an infinite vector. -

- -

- Following the lazy evaluation strategy, C_0 is not written to a static tape. It is implemented as a generator machine. The main evaluator queries this generator for its values up to the required extent \omega only as they are demanded. -

- -

- By definition, f(t)h(t) = 1. In the difference domain, this means the convolution of A_0 and C_0 must equal the identity vector I_0, where I_{0, 0} = 1 and all subsequent components are exactly zero. -

- -

- Recall the convolution formula for component k of the product: -

- - - I_{0, k} = \sum_{i=0}^{k} \sum_{j=0}^{k} A_{0, i} C_{0, j} \binom{k}{i} \binom{i}{k-j} - - - -

- To perform the deconvolution, we isolate the unknown component C_{0, k}. This term occurs in the summation strictly when j = k. When j = k, the term k-j equals 0, making the binomial coefficient \binom{i}{0} = 1. Factoring C_{0, k} out of the sum yields: -

- - - C_{0, k} \sum_{i=0}^{k} A_{0, i} \binom{k}{i} - - -

- A person familiar with Newton's forward difference formula will recognize that the summation \sum_{i=0}^{k} A_{0, i} \binom{k}{i} is exactly the evaluation of the original function at step k, or f(k). -

- -

- We can now solve for C_{0, k} recursively. For the base case k = 0, where I_{0, 0} = 1: -

- - - C_{0, 0} = \frac{1}{A_{0, 0}} - - -

- For all subsequent components where k > 0 and I_{0, k} = 0, we subtract the previously known terms of the convolution and divide by f(k): -

- -

- C_{0, k} = \frac{-1}{f(k)} \sum_{i=0}^{k} \sum_{j=0}^{k-1} A_{0, i} C_{0, j} \binom{k}{i} \binom{i}{k-j} -

- -

- This reveals a strict recurrent structure. To generate component k of the reciprocal vector, the generator machine relies entirely on the static components of the input polynomial A_0 and the previously computed components of the reciprocal C_{0, 0} through C_{0, k-1}. By encapsulating this recurrence within a generator, a programmer can perform exact division while maintaining finite memory bounds, extending the reciprocal vector only when the execution demands it. Note, this is a reciprocal of a function, rather than that of a value. -

- - - ---------- - - in the original Turing machine Architecture, add three tapes, stdin, stdout, and stderr diff --git a/document/book/scratchpad/TM-2026_parked.html b/document/book/scratchpad/TM-2026_parked.html new file mode 100644 index 0000000..62d4e89 --- /dev/null +++ b/document/book/scratchpad/TM-2026_parked.html @@ -0,0 +1,102 @@ + + more appendix mateial ... + +

Multiplication of synchronized difference vectors

+ +

+ Because the forward difference operator is linear, adding or subtracting two polynomials is achieved by the elementwise addition or subtraction of their initial difference vectors. Multiplication, however, requires a discrete convolution of the two vectors. +

+ +

+ Suppose a programmer has two initial difference vectors, A_0 representing function f(t) with an extent of \omega_a, and B_0 representing function g(t) with an extent of \omega_b. The goal is to compute the initial difference vector C_0 for the product function h(t) = f(t)g(t). +

+ +

+ Multiplying two polynomials of degrees \omega_a and \omega_b yields a polynomial of degree \omega_a + \omega_b. Therefore, the resulting vector C_0 will strictly have an extent of \omega_c = \omega_a + \omega_b, requiring a tape component count of \omega_a + \omega_b + 1. +

+ +

+ To determine the components of C_0 directly from A_0 and B_0 without evaluating the functions, we rely on the multiplication of their basis elements. In the calculus of finite differences, polynomials are expanded using binomial coefficients. The product of two binomial coefficients expands into a linear combination of higher binomial coefficients according to a known combinatorial identity: +

+ +

+ \binom{t}{i} \binom{t}{j} = \sum_{k=\max(i,j)}^{i+j} \binom{k}{i} \binom{i}{k-j} \binom{t}{k} +

+ +

+ By applying this identity across the summations of both input functions, the component k of the resulting vector C_0 can be computed algebraically. Each component C_{0, k} is the sum of the cross products of the input components, weighted by combinations of their indices: +

+ +

+ C_{0, k} = \sum_{i=0}^{\omega_a} \sum_{j=0}^{\omega_b} A_{0, i} B_{0, j} \binom{k}{i} \binom{i}{k-j} +

+ +

+ Thus, while a Turing Machine extending the function only requires a simple accumulator, a machine tasked with multiplying two initial tapes must perform a combinatorial cross multiplication to generate the expanded tape before the extension sequence can begin. +

+ +

Division and the Reciprocal Difference Vector

+ +

+ If the multiplication of two polynomials in the finite difference domain is a discrete convolution, then division is a discrete deconvolution. By finding the reciprocal of a difference vector, a programmer can perform division using the same combinatorial architecture. +

+ +

+ Let A_0 be the initial difference vector for a polynomial f(t). We seek the reciprocal difference vector C_0, which represents the function h(t) = 1/f(t). Because the reciprocal of a polynomial is a rational function, its forward differences will never reduce to zero. Thus, C_0 is an infinite vector. +

+ +

+ Following the lazy evaluation strategy, C_0 is not written to a static tape. It is implemented as a generator machine. The main evaluator queries this generator for its values up to the required extent \omega only as they are demanded. +

+ +

+ By definition, f(t)h(t) = 1. In the difference domain, this means the convolution of A_0 and C_0 must equal the identity vector I_0, where I_{0, 0} = 1 and all subsequent components are exactly zero. +

+ +

+ Recall the convolution formula for component k of the product: +

+ + + I_{0, k} = \sum_{i=0}^{k} \sum_{j=0}^{k} A_{0, i} C_{0, j} \binom{k}{i} \binom{i}{k-j} + + + +

+ To perform the deconvolution, we isolate the unknown component C_{0, k}. This term occurs in the summation strictly when j = k. When j = k, the term k-j equals 0, making the binomial coefficient \binom{i}{0} = 1. Factoring C_{0, k} out of the sum yields: +

+ + + C_{0, k} \sum_{i=0}^{k} A_{0, i} \binom{k}{i} + + +

+ A person familiar with Newton's forward difference formula will recognize that the summation \sum_{i=0}^{k} A_{0, i} \binom{k}{i} is exactly the evaluation of the original function at step k, or f(k). +

+ +

+ We can now solve for C_{0, k} recursively. For the base case k = 0, where I_{0, 0} = 1: +

+ + + C_{0, 0} = \frac{1}{A_{0, 0}} + + +

+ For all subsequent components where k > 0 and I_{0, k} = 0, we subtract the previously known terms of the convolution and divide by f(k): +

+ +

+ C_{0, k} = \frac{-1}{f(k)} \sum_{i=0}^{k} \sum_{j=0}^{k-1} A_{0, i} C_{0, j} \binom{k}{i} \binom{i}{k-j} +

+ +

+ This reveals a strict recurrent structure. To generate component k of the reciprocal vector, the generator machine relies entirely on the static components of the input polynomial A_0 and the previously computed components of the reciprocal C_{0, 0} through C_{0, k-1}. By encapsulating this recurrence within a generator, a programmer can perform exact division while maintaining finite memory bounds, extending the reciprocal vector only when the execution demands it. Note, this is a reciprocal of a function, rather than that of a value. +

+ + + ---------- + + in the original Turing machine Architecture, add three tapes, stdin, stdout, and stderr diff --git a/shared/tool/env_to_emacs b/shared/tool/env_to_emacs new file mode 100755 index 0000000..947efc0 --- /dev/null +++ b/shared/tool/env_to_emacs @@ -0,0 +1,30 @@ +#!/usr/bin/env bash +script_afp=$(realpath "${BASH_SOURCE[0]}") + +set -euo pipefail + +if [ -z "${REPO_HOME:-}" ]; then + echo "emacs-sync:: error: REPO_HOME is not set. Source a setup environment first." + exit 1 +fi + +if ! command -v emacsclient >/dev/null 2>&1; then + echo "emacs-sync:: error: emacsclient is not installed or not in PATH." + exit 1 +fi + +# Push variables to the Emacs global environment +emacsclient --eval "(setenv \"REPO_HOME\" \"$REPO_HOME\")" > /dev/null +emacsclient --eval "(setenv \"PROJECT\" \"$PROJECT\")" > /dev/null + +if [ -n "${ROLE:-}" ]; then + emacsclient --eval "(setenv \"ROLE\" \"$ROLE\")" > /dev/null +fi + +# Sync the PATH so Emacs subprocesses find the local tools +emacsclient --eval "(setenv \"PATH\" \"$PATH\")" > /dev/null + +# Update exec-path for native Emacs executable resolution +emacsclient --eval "(setq exec-path (append (parse-colon-path \"$PATH\") (list exec-directory)))" > /dev/null + +echo "Emacs environment synchronized for project: $PROJECT" diff --git a/sys b/sys new file mode 100644 index 0000000..e69de29 diff --git a/tempfile b/tempfile new file mode 100644 index 0000000..e69de29