From: Thomas Walker Lynch
- Generally it is more efficient to perform arithmetic in the first order. Also, it is easier to write Turing Machine control programs if we take Natural Number arithmetic as already available, say, from a subroutine library. Where second order computation becomes useful is in places where a result cannot be computed.
+ Generally it is more efficient to perform arithmetic in the first order. Also, it is easier to write Turing Machine control programs if we take Natural Number arithmetic as already available, say, from a subroutine library. On processors fixed word length arithmetic is built into the hardware. Where second order computation becomes useful is in places where a result cannot be computed in the first order.
@@ -1139,11 +1139,11 @@
- There is no special help here. Hence, remains as an error flag.
+ There is no special help here. Hence, remains as an error flag.
- The divide machine cannot be run to produce a value, as any value from the field assigned to it would lead to contradictions. For other values, operations, when given an and a value, there is only one possible value, and it can be recovered with . However, when is zero, and only when it is zero, is solely determined by independent of , so is ignored; it could be any value. There is no way to recover it from .
+ The divide machine cannot be run to produce a value, as any value from the field assigned to it would lead to contradictions. Stated more precisely, for operations, when given an and a value, there is only one possible value, and it can be recovered with . However, when is zero, and only when it is zero, is solely determined by independent of , so is ignored; it could be any value. There is no way to recover it from .
The multiplicative inverse of the additive identity
Evaluating and Extending
@@ -1157,15 +1157,15 @@
- In a variation of analytical evaluation, we can instead mount an initial value for a function, and call a machine to repeatedly the function. To state this more precisely, when a machine tape is given the initial value of on its input tape, and then when run until it halts, writes the result , then running the machine extends the function. Typically extension machines are designed so they can be used recurrently to generate a sequence of function values, . + In a variation of analytical evaluation, we can instead mount an initial value for a function, and call a machine to repeatedly the function. To state this more precisely, when a machine tape is given the initial value of on its input tape, and then when run until it halts having written the result to the tape, we say that running the machine extends the function. Typically extension machines are designed so they can be used recurrently to extend the function indefinitely, thus producing a sequence of function values, .
- The difference between function values in the above sequence is called the first finite difference. The forward first difference is defined as: + The difference between adjacent function values in the above sequence is called the first finite difference. The forward first difference is defined as: . While the backwards first difference is . - If 'forward' or 'backwards' is not specified, then the difference is taken to be 'forward'. Hence, extending a function is identical to adding the first difference. + If 'forward' or 'backwards' is not specified, then the difference is taken to be 'forward'. Hence, extending a function is identical to adding the first difference. Thus a sequence of function values, implies a sequence of first differences.
@@ -1224,11 +1224,11 @@
- In order to design for recurrent extensions, the Turing Machine tape must retain the current function value alongside its forward differences. Because the second difference is constant, the tape requires three logical cells of state to compute the next step: . No matter how long the finite difference table becomes, each row will have three entries. + In order to design for indefinite extension, the Turing Machine tape must retain the function value alongside with its forward differences. Because the second difference is constant higher order differences are all zero. Hence the tape requires a three component vector to hold the differences.
- Here is the sequence of tape states as the machine is repeatedly called to extend the function from its initial conditions at . During each step, the machine adds to , and to : + Here is the sequence of tape states as the machine is repeatedly called to extend the function from its initial conditions at . During each step, the machine adds to , and to , as there is no , it is taken to be zero, so is merely copied down.
- For Call 1, the input tape is the Initial Tape. In Call 1 the machine will take the function value on the input tape, 0, and add the first difference, writing the function value to the result tape. The machine will then continue on and take the first difference on the input tape, 1, and add the second difference on the input tape, 2, writing the first difference for the result tape, 3. Continuing on, the machine will not find a third difference on the input tape, so it will copy the second difference found on the input tape as the second difference for the result tape. + For Call 1, the input tape is the Initial Tape. In Call 1 the machine will take the function value on the input tape, 0, and add the first difference, 1, writing the result 1, to the result tape. The machine will then continue on and take the first difference on the input tape, 1, and add the second difference on the input tape, 2, writing the result first difference to the result tape, 3. Then finishing up, the machine will not find a third difference on the input tape, so it will copy the second difference found on the input tape as the second difference for the result tape.
- For Call 2, the input tape is the result tape from Call 1. Execution then proceeds identically to Call 1. This pattern repeats for all successive calls. A person can see the values of as the first number on each result tape. + For Call 2, the input tape is the result tape from Call 1. Execution then proceeds identically to Call 1. This pattern repeats for all successive calls. A person can see the values of as the first number on each result tape. Any polynomial can be computed in this manner, and the only ALU required is an adder that is also capable of subtraction.
- The extension function can be called any number of times when the initial tape is identical to row 0 of the finite difference table. No other row is needed from the table. + The extension function can be called any number of times when the initial tape is identical to row 0 of the table of finite differences. No other row is needed from the table of finite differences.
- This follows from the fact that the extension function is performing the exact operations that were performed when we generated the difference table. The extension function is merely formalizing the rules used for generating the table. As a consequence, the result tape after the Nth call is identical to row N of the table. + This follows from the mechanical procedure given above for computing the next row of the table from any given row.
++ Let the refer to the maximum index for accessing a component of a vector; the extent is often denoted as . Performing extensions to calculate the function value at call makes use of the initial tape up to index , or as many nonzero values as are available up to that extent. +
- Let the refer to the maximum index for accessing a component of a vector; the extent is often denoted as . Performing extensions to calculate the function value at call will make use of the initial tape up to index , or as many nonzero values as are available up to that extent. + To make this precise, let us define the at call as . The components of this vector are up to the maximum available difference. The first component, , is the function value itself. The component is the first difference, and so on.
- To make this precise, let us define the at call as . The components of this vector are , where is the maximum available difference. The first component, , is the function value itself. The component is the first difference, and so on. + Thus, this lemma claims that to compute , the highest component index accessed from the initial tape vector will be .
- +- The extension operation follows a simple recurrence relation for any component : . + Note that to compute the result by the mechanical procedure given above requires adding the input tape first difference to the prior difference. That is, . Hence we see immediately the lemma holds for the first call where .
- We can now prove by induction on the call index that computing requires exactly the components through from the initial tape . + For an arbitrary value we discovered that the highest index required of the vector is . + The value is computed from our mechanical procedure as + . Given we already know that to compute the value merely requires the prior value and first difference, we find that the largest index needed from vector is 2.
- Base Case (k=0): To generate the value at call 0, , the machine performs zero extensions. It accesses only the value at index 0 of the initial tape. The required extent is 0. + There is a strict linear progression here. The computation of any component inherently requires and . Therefore, each step backwards in time expands the required index by exactly one. By induction, computing the base value at call requires tracing back steps to the initial tape at , which forces the maximum accessed index to be . Thus, the highest required component from the initial tape is .
- Inductive Hypothesis: Assume that generating the value at call , which is , requires the initial tape components through . + The exact algebraic composition of this mechanical expansion is formalized by Newton's calculus of finite differences . Instead of executing the machine incrementally, a person can calculate the function value at call directly as a linear combination of the initial tape components using Newton's forward difference formula:
- Inductive Step: Consider the requirement for the value at call , which is . By the extension rule, . +
-+ Because the binomial coefficient evaluates to exactly zero for any integer , the summation naturally truncates at index . This algebraic property perfectly mirrors the physical boundary established by the machine execution trace. Furthermore, the relationship is symmetric. A person can compute the specific components of the initial tape, , directly from the sequence of evaluated function calls, , using the alternating binomial sum: +
+- The union of the dependencies for and spans from index 0 to index . Thus, generating the value at call requires the initial tape to have an extent of exactly . The lemma holds. +
- This proof applies regardless of whether the finite difference table is infinitely deep or truncated. If a person evaluates a tenth degree polynomial but only programs the machine to perform two calls, the initial tape only needs an extent of 2, utilizing components 0, 1, and 2. Each extension consumes one cell of state. If a programmer intends to execute the extension function to reach call , the initial tape must have an extent of , meaning it holds values. + As a consequence of this lemma, we know that for any finite number of calls, i.e. finite , that a finite prefix of is used.
-A polynomial of degree is defined by a highest order term . The first forward difference operator, , inherently cancels the term. The binomial expansion of yields as its leading term, which subtracts out, leaving a new polynomial of exactly degree . @@ -1309,48 +1315,163 @@ Taking successive finite differences reduces the degree of the polynomial by exactly one at each step. Consequently, the th difference evaluates to a constant, and the next difference evaluates to exactly zero.
+ + +- Because component 0 of the initial difference vector, , holds the function value itself, and the differences occupy components 1 through , the initial vector requires a component count of , where . This defines a vector with a maximum index, or extent, of . + If an initial difference vector has an extent of , the sequence of function values generated by the machine corresponds to a polynomial of exactly degree . A person can demonstrate this by algebraically recovering the polynomial constants, , directly from the initial tape components, .
-+ Consider an extent of , forming a polynomial of degree 0, . Evaluating at yields . Because the machine's initial value is , a person establishes . +
- If an initial difference vector has an extent of (meaning the component is constant and all higher differences are zero), the sequence of function values can be generated by a polynomial of degree . + When a person extends the extent to , the next term is added to form . The first forward difference at is . Therefore, . Note that the addition of the linear term did not change the evaluation at ; the constant remains firmly anchored to .
- This is proven by constructing the function directly from the initial tape utilizing Newton's forward difference formula. The function value at any discrete step is given by a linear combination of the initial difference components multiplied by binomial coefficients: + Extending to , the polynomial becomes . The second difference at is calculated as . Substituting the polynomial yields . Thus, a person recovers . The lower coefficient then adjusts to .
- + Continuing to , the polynomial is . Evaluating the third difference isolates the highest order term, yielding . This allows immediate recovery of the highest constant: .
- To determine the degree of this constructed function, a person must evaluate the binomial coefficient, . It expands into a polynomial in : + Once is known, the shifting nature of the lower order constants becomes apparent through back substitution. The second difference equation expands to . Substituting the known allows recovery of . Similarly, the first difference equation expands to , which resolves to . Through all of this shifting, remains entirely unperturbed.
- + To observe the formal mechanics of this progression, a person can array these relationships into a matrix equation mapping the polynomial constants, , to the initial tape differences, . The coefficients of this transformation are defined by the Stirling numbers of the second kind, denoted :
+ +- Because the numerator contains factors of , the term is a polynomial of exactly degree . + Because each successive forward difference operator annihilates the lowest power of , the resulting matrix is strictly upper triangular. For any extent , the final row simplifies to . Because an initial tape of extent strictly dictates that is nonzero, is guaranteed to be nonzero.
- The initial tape components, , are static scalar constants. Therefore, the summation consists of adding together polynomials of increasing degrees from up to . Because the extent is , the scalar is nonzero, ensuring that the highest degree term in the summation does not vanish. Thus, the constructed function is strictly a polynomial of degree . + The main diagonal contains strictly non-zero factorials, ensuring the matrix is invertible. By inverting this matrix, a person replaces the cascading back substitution with a direct, closed-form equation to recover any constant . The inversion utilizes the signed Stirling numbers of the first kind, denoted (see the Appendix on Stirling numbers).
++ +
++ Thus, extending this procedure times definitively recovers the constants for exactly a polynomial of degree . +
++ This mechanical recovery of standard polynomial constants is completely analogous to Newton's interpolation formula . Instead of resolving the standard constants through an upper triangular matrix, a person can construct the polynomial directly by treating the initial tape components as the exact coefficients for a basis of binomial terms: +
+ ++ +
+ ++ Because the th binomial coefficient expands into a polynomial of exactly degree , and the summation is bounded by the finite extent where is definitively nonzero, the constructed function is guaranteed to be a polynomial of degree . +
+ ++ If an initial difference vector has an extent of , the sequence of function values generated by the machine corresponds to a polynomial of exactly degree . A person can demonstrate this by algebraically recovering the polynomial constants, , directly from the initial tape components, . +
++ Consider an extent of , forming a polynomial of degree 0, . Evaluating at yields . Because the machine's initial value is , a person establishes . +
+ ++ When a person extends the extent to , the next term is added to form . The first forward difference at is . Therefore, . Note that the addition of the linear term did not change the evaluation at ; the constant remains firmly anchored to . +
+ ++ Extending to , the polynomial becomes . The second difference at is calculated as . Substituting the polynomial yields . Thus, a person recovers . The lower coefficient then adjusts to . +
+ ++ Continuing to , the polynomial is . Evaluating the third difference isolates the highest order term, yielding . This allows immediate recovery of the highest constant: . +
+ ++ Once is known, the shifting nature of the lower order constants becomes apparent through back substitution. The second difference equation expands to . Substituting the known allows recovery of . Similarly, the first difference equation expands to , which resolves to . Through all of this shifting, remains entirely unperturbed. +
+ ++ To observe the structural integrity of this progression, a person can array these relationships into a system of equations mapping the initial tape to the polynomial constants: +
+ +| Difference | +Equation | +
|---|---|
| + | + |
| + | + |
| + | + |
| + | + |
| + | + |
| + | + |
+ A definitive pattern emerges. Because each successive forward difference operator annihilates the lowest power of , the resulting algebraic system is strictly upper triangular. For any extent , the th difference equation reduces to . This structural guarantee permits a person to reliably recover . Because an initial tape of extent strictly dictates that is nonzero, is guaranteed to be nonzero. All subsequent lower order constants are then systematically resolved through cascading back substitution. Thus, extending this procedure times definitively recovers the constants for exactly a polynomial of degree . +
+ ++ This mechanical recovery of standard polynomial constants is completely analogous to Newton's interpolation formula . Instead of resolving the standard constants through an upper triangular matrix, a person can construct the polynomial directly by treating the initial tape components as the exact coefficients for a basis of binomial terms: +
+ ++ +
+ ++ Because the th binomial coefficient expands into a polynomial of exactly degree , and the summation is bounded by the finite extent where is definitively nonzero, the constructed function is structurally guaranteed to be a polynomial of degree . +
- Here is the finite difference table for the function . + Here is the table of finite differences for the function .
| Count () | - | 1st Diff () | -2nd Diff () | -3rd Diff () | +1st Diff () | +2nd Diff () | +3rd Diff () |
|---|
- The first difference of is also , so the first row of the table will have an infinite number of values. Fortunately, due to the "Lemma, for the first N values, the first row need only have N + 1 columns" only N+1 of these values are needed for the Nth call of the recurrence function. Furthermore, when new values are needed, they are easily computed. We need not store the first row of the difference table on the tape. + The first difference of is also , so row 0 of the table will have an infinite number of values. Fortunately, due to the lemma stating that evaluating call requires an initial tape populated with components 0 through of row 0, exactly those components are needed for the recurrence to proceed. Furthermore, when new values are needed, they are easily computed. A programmer need not store the entire infinite first row of the difference table on a static tape. +
+ ++ Instead of attempting to write an infinite sequence to a physical tape, a person can encapsulate the generation logic within a dedicated Turing Machine. In the tradition of constructive arithmetic associated with Boehm, this generator machine acts as a virtual tape. When the primary extension machine requires the next difference component, it queries the generator machine, which computes and yields the value on demand. This lazy evaluation strategy elegantly handles functions with infinite difference sequences without exhausting finite memory.
Performing operations on with recurrence functions requires integrating a call algebra in addition to integrating the machines. Often this is handled by consider the machines being composed to be subroutines, and the outer controller than explicitly calls them. The call algebra can then be dynamic depending on the input. Calls can be skipped due to being inside conditionals, or performed an arbitrary number of times in loops. Though simple control structures that are static are more math like, and are more likely to be more effected by optimizations (simplifications).
+ ++ Because the forward difference operator is linear, adding or subtracting two polynomials is achieved by the elementwise addition or subtraction of their initial difference vectors. Multiplication, however, requires a discrete convolution of the two vectors. +
+ ++ Suppose a programmer has two initial difference vectors, representing function with an extent of , and representing function with an extent of . The goal is to compute the initial difference vector for the product function . +
+ ++ Multiplying two polynomials of degrees and yields a polynomial of degree . Therefore, the resulting vector will strictly have an extent of , requiring a tape component count of . +
+ ++ To determine the components of directly from and without evaluating the functions, we rely on the multiplication of their basis elements. In the calculus of finite differences, polynomials are expanded using binomial coefficients. The product of two binomial coefficients expands into a linear combination of higher binomial coefficients according to a known combinatorial identity: +
+ ++ +
+ ++ By applying this identity across the summations of both input functions, the component of the resulting vector can be computed algebraically. Each component is the sum of the cross products of the input components, weighted by combinations of their indices: +
+ ++ +
+ ++ Thus, while a Turing Machine extending the function only requires a simple accumulator, a machine tasked with multiplying two initial tapes must perform a combinatorial cross multiplication to generate the expanded tape before the extension sequence can begin. +
+ ++ If the multiplication of two polynomials in the finite difference domain is a discrete convolution, then division is a discrete deconvolution. By finding the reciprocal of a difference vector, a programmer can perform division using the same combinatorial architecture. +
+ ++ Let be the initial difference vector for a polynomial . We seek the reciprocal difference vector , which represents the function . Because the reciprocal of a polynomial is a rational function, its forward differences will never reduce to zero. Thus, is an infinite vector. +
+ ++ Following the lazy evaluation strategy, is not written to a static tape. It is implemented as a generator machine. The main evaluator queries this generator for its values up to the required extent only as they are demanded. +
+ ++ By definition, . In the difference domain, this means the convolution of and must equal the identity vector , where and all subsequent components are exactly zero. +
+ ++ Recall the convolution formula for component of the product: +
+ ++ +
+ ++ To perform the deconvolution, we isolate the unknown component . This term occurs in the summation strictly when . When , the term equals 0, making the binomial coefficient . Factoring out of the sum yields: +
+ ++ +
+ ++ A person familiar with Newton's forward difference formula will recognize that the summation is exactly the evaluation of the original function at step , or . +
+ ++ We can now solve for recursively. For the base case , where : +
+ ++ +
+ ++ For all subsequent components where and , we subtract the previously known terms of the convolution and divide by : +
+ ++ +
+ ++ This reveals a strict recurrent structure. To generate component of the reciprocal vector, the generator machine relies entirely on the static components of the input polynomial and the previously computed components of the reciprocal through . By encapsulating this recurrence within a generator, a programmer can perform exact division while maintaining finite memory bounds, extending the reciprocal vector only when the execution demands it. Note, this is a reciprocal of a function, rather than that of a value. +
+The row 0 vector for the function is:
+ + + +The row 0 vector for is:
+ + + +We will place these functions into a single system, so we need to synchronize them. Let us assume that will be called 3 times each time that is called once. Then it becomes:
+ + + +It still starts in the same place, but every 3 steps it jumps by 3 rather than the 1 it jumps per step.
+ +Consider the quotient:
+ + +The two synchronized machines that are called against this quotient can be simplified to a rational generator machine. This machine runs the two initial difference vectors in parallel and evaluates their ratio dynamically at each step, rather than attempting to compute a static deconvolution vector upfront.
+ +By applying the discrete Leibniz rule, a person can simplify the heavy combinatorial deconvolution into a highly optimized linear recurrence relation. Let be the component of the numerator and be the component of the quotient. The relation to find the next quotient component reduces cleanly to:
+ + + +Using this recurrence, the machine will successfully generate quotient values, but on call 5 there is again a .
+ +------
- +| Count () | +Quotient () | +1st Diff Ratio () | +2nd Diff Ratio () | +
|---|---|---|---|
| 0 | +31/15 | +1/3 | ++ |
| 1 | +30/12 | +2/3 | ++ |
| 2 | +28/9 | +4/3 | ++ |
| 3 | +24/6 | +8/3 | ++ |
| 4 | +16/3 | +16/3 | ++ |
| 5 | ++ | 32/3 | ++ |
| 6 | +32/3 | +64/3 | ++ |
| 7 | +96/6 | +128/3 | ++ |
| 8 | +224/9 | +256/3 | ++ |
| 9 | +480/12 | +512/3 | ++ |