CS 70 Discrete Mathematics for CS

Fall 2004  Rao  Lecture 3

This lecture covers further variants of induction, including strong induction and the closely related wellordering axiom. We then apply these techniques to prove properties of simple recursive programs.

Strong induction

Axiom 3.1 (Strong Induction): For any property P, if P(0) and ∀n∈ N (P(0) ∧ P(1) ∧ ...∧ P(n) ⟹ P(n + 1)), then ∀n∈ N P(n).

This says that if all the following sentences are true:

P(0)
P(0) ⟹ P(1)
P(0) ∧ P(1) ⟹ P(2)
P(0) ∧ P(1) ∧ P(2) ⟹ P(3)
P(0) ∧ P(1) ∧ P(2) ∧ P(3) ⟹ P(4)

and so on, then P(n) must be true for all n. Intuitively, this seems quite reasonable. If the truth of P all the way up to n always implies the truth of P(n + 1), then we immediately obtain the truth of P all the way up to n + 1, which implies the truth of P(n + 2), and so on ad infinitum.

If we compare the Strong Induction axiom to the original Induction axiom from Lecture 2, we see that Strong Induction appears to make it easier to prove things. With simple induction, one must prove P(n + 1) given the inductive hypothesis P(n); with strong induction one gets to assume the inductive hypothesis P(0) ∧ P(1) ∧ ...∧ P(n), which is much stronger.

Consider the following example, which is one half of the Fundamental Theorem of Arithmetic. (The other half says that the product is unique.)

Theorem 3.1: Any natural number n > 1 can be written as a product of primes.

To prove this, of course, we need to define prime numbers:

Definition 3.1 (Prime): A natural number n > 1 is prime iff it has exactly two factors (1 and n). 1 itself is not prime.

Let’s see first what happens when we try a simple induction:

Proof: (Attempt 1) The proof is by induction over the natural numbers n > 1.

• Base case: prove P(2).

P(2) is the proposition that 2 can be written as a product of primes. This is true, since 2 can be written as the product of one prime, itself. (Remember that 1 is not prime!)

• Inductive step: prove P(n) ⟹ P(n + 1) for all natural numbers n > 1.

1. The inductive hypothesis states that n can be written as a product of primes.

2. To prove: n + 1 can be written as a product of primes.

3. We’re stuck: given P(n), we could easily establish P(2n) or P(7n), but P(n + 1) is unconnected to P(n).

□

With a strong induction, we can make the connection between P(n+1) and earlier facts in the sequence that are relevant. For example, if n + 1= 72, then P(36) and P(24) are useful facts.

Proof: The proof is by strong induction over the natural numbers n > 1.

• Base case: prove P(2), as above.

• Inductive step: prove P(2) ∧ ...∧ P(n) ⟹ P(n + 1) for all natural numbers n > 1.

1. The inductive hypothesis states that, for all natural numbers m from 2 to n, m can be written as a product of primes.

2. To prove: n + 1 can be written as a product of primes.

3. Proof by cases:

– n + 1 is prime: then n + 1 can be written as the product of one prime, itself.

– n + 1 is not prime: then by the definition of prime numbers, there exist integers a, b such that 2 ≤ a,b < n + 1 and n + 1= a · b. By the inductive hypothesis, both a and b can be written as a product of primes. Hence n + 1 can be written as a product of primes.

□

Consider the following example, which is of immense interest to post offices and their customers:

Theorem 3.2: Any integer amount of postage from 8¢ upwards can be composed from 3¢ and 5¢ stamps.

With a strong induction, we can make the connection between P(n + 1) and earlier facts in the sequence. In particular, P(n−2) is relevant because n+1 can be composed from the solution for n−2 plus one 3¢ stamp. So the inductive step works if P(n − 2) is known already. This will not be the case when n + 1 is 9 or 10, so we will need to handle these separately.

Proof: The proof is by strong induction over the natural numbers n ≥ 8.

• Base case: prove P(8).

P(8) is the proposition that 8¢ of postage can be composed from 3¢ and 5¢ stamps. This is true, requiring 1 of each.

• Inductive step: prove P(8) ∧ ...∧ P(n) ⟹ P(n + 1) for all natural numbers n ≥ 8.

1. The inductive hypothesis states that, for all natural numbers m from 8 to n, m¢ of postage can be composed from 3¢ and 5¢ stamps.

2. To prove: (n + 1)¢ of postage can be composed from 3¢ and 5¢ stamps.

3. The cases where n + 1 is 9 or 10 must be proved separately. 9¢ can be composed from three 3¢ stamps. 10¢ can be composed from two 5¢ stamps.

4. For all natural numbers n + 1 > 10, the inductive hypothesis entails the proposition P(n − 2). If (n − 2)¢ can be composed from 3¢ and 5¢ stamps, then (n + 1)¢ can be composed from 3¢ and 5¢ stamps simply by adding one more 3¢ stamp.

□

Notice that, as with the tiling problem, the inductive proof leads directly to a simple recursive algorithm for selecting a combination of stamps.

Notice also that a strong induction proof may require several “special case” proofs to establish a solid foundation for the sequence of inductive steps. It is easy to overlook one or more of these.

Simple induction and strong induction

We have seen that strong induction makes certain proofs easy even when simple induction appears to fail. A natural question to ask is whether the strong induction axiom is in fact logically stronger than the simple induction axiom; if so, then the theorems that can be proved using strong induction are a strict superset of the theorems that can be proved using simple induction.

Let’s investigate this question. First, does the strong induction entail the simple induction axiom? Intuitively, this seems to be true. Let’s put the two axioms side by side and examine their structure (we’ll take the restriction to the natural numbers as implicit here):

Simple:  P(0) ∧ [∀n P(n) ⟹ P(n + 1)]  ⟹ ∀n P(n)
Strong: P(0) ∧ [∀n P(0) ∧ ...∧ P(n) ⟹ P(n + 1)] ⟹ ∀n P(n)

We can reduce this to the following basic form (with the obvious definitions for propositions A, B, B′, and C):

Simple: A ∧ B ⟹ C
Strong: A ∧ B′ ⟹ C

Now if P(n) ⟹ P(n + 1), then P(0) ∧ ...∧ P(n) ⟹ P(n + 1). Hence, B ⟹ B′(i.e., b is stronger than B′). Hence, if A ∧ B′suffice to prove C, then surely the stronger fact A ∧ B also suffices to prove C. (This is easily checked using truth tables.) Therefore, the strong induction axiom entails the simple induction axiom.

Second, does the simple induction entail the strong induction axiom? One might expect not, but in fact it does! We can see this by defining, for any property P(n), the proposition

Q(n) ⇔ P(0) ∧ ...∧ P(n)

That is, Q(n) is the property “P holds from 0 to n.” The idea is that simple induction using Q is in fact identical to strong induction using P. The simple induction axiom for Q is

Q(0) ∧ [∀n Q(n) ⟹ Q(n + 1)] ⟹ ∀n Q(n)

Expanding out the definition of Q, we obtain

P(0) ∧ [∀n (P(0) ∧ ...∧ P(n)) ⟹ (P(0) ∧ ...∧ P(n) ∧ P(n + 1))] ⟹ [∀n (P(0) ∧ ...∧ P(n))]

A few moments’ thought [we recommend thinking this thought yourself] reveals that this proposition is logically equivalent to the proposition

P(0) ∧ [∀n P(0) ∧ ...∧ P(n) ⟹ P(n + 1)] ⟹ ∀n P(n)

which is the strong induction axiom. Therefore we have shown (rather informally perhaps) the following:

Theorem 3.3: The strong induction axiom and the simple induction axiom are logically equivalent.

Why have two different forms of induction then? The point is that strong induction reminds its user of the opportunity to use P(0) ∧ ...∧ P(n) in the inductive step when P(n) is defined the “natural” way from the statement of the theorem to be proved.

The well-ordering principle

If one thinks about why induction works, one might ask the question “How could the induction axiom fail to be true?” To violate the induction axiom, we would need to satisfy its antecedent (so P(0) is true and P(n) ⟹ P(n + 1) for all n) while violating its consequent (so ∃n¬P(n)). Let us consider the first n for which P(n) is false. By definition, we know that P(n − 1) is true; and by assumption we know that P(n − 1) ⟹ P(n); therefore we have a straightforward contradiction!

Have we proved the induction axiom? Actually, no; we have proved that the induction axiom follows from another axiom, which was used implicitly in defining “the first n for which P(n) is false.”

Axiom 3.2 (Well-Ordering): Every nonempty set of natural numbers has a smallest element.

Duh. Doesn’t every nonempty set of orderable elements have a smallest element? No! Every finite set has a smallest element, but not every infinite set. For example, neither the integers nor even the positive rationals have a smallest element.

The well-ordering principle not only underlies the induction axioms, but also has direct uses in its own right. A particularly elegant example concerns the existence of cycles in tournaments.

Definition 3.2 (Round-Robin): A round-robin tournament is one in which each player p plays each other player q exactly once and either wins (p ≻ q) or loses (q ≻ p).

Definition 3.3 (Cycle): A cycle in a tournament is a set of players {p_{1}...p_{k}} such that p_{1} ≻ p_{2} ≻ ... ≻ p_{k−1} ≻ p_{k} ≻ p_{1}.

Theorem 3.4: In every round-robin tournament, if there is a cycle, then there is a cycle of length 3.

Proof: The proof is by contradiction.

1. Assume the theorem is false. Consider the set of cycle lengths of the tournament. By assumption, this must be nonempty.

2. By the well-ordering principle, it must have a smallest element k. By assumption, k > 3.

3. Let the first three elements in this cycle be p_{1}, p_{2}, p_{3}, and consider the result of the match between p_{1} and p_{3}.

4. Case 1: p_{1} ≻ p_{3}. Then we have p_{1} ≻ p_{3} ≻ ... ≻ p_{k−1} ≻ p_{k} ≻ p_{1}, i.e., a cycle of length k − 1, contradicting our assumption that the smallest cycle has length k > 3.

5. Case 2: p_{3} ≻ p_{1}. Then we have p_{1} ≻ p_{2} ≻ p_{3} ≻ p_{1}, i.e., a cycle of length 3, contradicting our assumption that the smallest cycle has length k > 3.

6. By the definition of round-robins, either p_{1} ≻ p_{3} or p_{3} ≻ p_{1}. Therefore, a contradiction exists.

7. Hence, it must be the case that any tournament with a cycle has a cycle of length 3.

□ This proof illustrates a common way to use well-ordering combined with proof by contradiction. The wellordering principle allows one to focus on a concrete counterexample with the property that every smaller example satisfies some property. For certain proofs, this can be an easier thought process than induction.

Induction and recursion

There is an intimate connection between induction and recursion. Essentially every recursive function relies for its correctness on an inductive proof. Remember that a recursive function applies itself to a “smaller” argument. The inductive proof says that if the recursive function works on all smaller arguments it will work on the current argument.

We’ll begin with that old favourite, the factorial function. Let’s give a recursive definition for a function f(n) and show it’s identical to n!. For any n∈ N,

f(n) = 1 if n = 0
f(n) = nf(n − 1) otherwise

Theorem 3.5: For all natural numbers n, f(n) = n!.

Proof: The proof is by induction over the natural numbers. Let P(n) be the proposition that f(n) = n!.

• Base case: prove P(0).

P(0) is the proposition that f(0) = 0!. By the definition above, f(0) = 1 = 0!, hence P(0) is true.

• Inductive step: prove P(n) ⟹ P(n + 1) for all n∈ N.

1. The inductive hypothesis is f(n) = n!.

2. To prove: f(n + 1) = (n + 1)!.

3. By the definition above,

f(n + 1) = (n + 1) · f(n) because n∈ N so (n + 1) ≠ 0
= (n + 1) · n! by the inductive hypothesis
= (n + 1) · n · (n − 1)··· 1 = (n + 1)!

Hence, by the induction principle, ∀n∈ N f(n) = n!. □

Mathematical functions and real programs

The above discussion applies to a purely mathematical definition of f(n). If we wanted to reason about a real program, first we have to write it in a real language, such as Scheme:

(define (factorial n)
  (if (= n 0)
       1
     (* n (factorial (- n 1)))))

The statement of correctness for a program is not quite so straightforward as for a mathematically defined function:

Theorem 3.6: For all (computer representations of) natural numbers n, the result of evaluating the expression (factorial n) is the computer representation of n!.

The proof that the program is correct is very similar to the proof that the recursive function has the right values. Notice the notation: actual syntactic elements of the programming language are in typewriter font, while variables that range over them are in italics.

Proof: The proof is by induction over the natural numbers. Let P(n) be the proposition that (factorial n) = n!.

• Base case: prove P(0).

P(0) is the proposition (factorial 0) = 0!. By the definition above,

          (factorial 0)

           = (if (= 0 0) 1 (* 0 (factorial (- 0 1))))

= 1 by evaluation of if

• Inductive step: prove P(n) ⟹ P(n + 1) for all n∈ N.

1. The inductive hypothesis is (factorial n) = n!.

2. To prove: (factorial (n + 1)) = (n + 1)!.

3. By the definition above,

              (factorial (n + 1))

               = (if (= (n + 1) 0) 1 (* (n + 1) (factorial (- (n + 1) 1))))

= (* (n + 1) (factorial (- (n + 1) 1)))  because n∈ N so (n + 1) ≠ 0

               = (* (n + 1) (factorial n))

= (* (n + 1) n!) by the inductive hypothesis
= (n + 1)!

Hence, by the induction principle, ∀n∈ N (factorial n) = n! □

Notes on this proof:

• We appeal implicitly to several aspects of the evaluation of programs such as the binding of parameters, the definition of if-expressions, and the correspondence between the mathematical function “−” and the built-in function “-”. These lemmata are an essential part of the definition of the programming language and can be stated and proved once and for all.

• The theorem as stated is almost certainly false! A real proof of correctness, for a suitably reduced theorem, must handle the important differences between mathematical entities and the corresponding entities in the computer. For example, n! is well-defined for any natural number, but (factorial n) fails if n is large enough to cause an overflow in integer multiplication. Another way to say this is that * is not the same as the mathematical multiplication function.

• As defined, (factorial n) causes an error for nonnumeric, noninteger, or negative inputs. (What error arises from negative inputs?) A full specification for a really robust system should lay out the correct responses to all possible inputs.

For the most part, we will use mathematical rather than Scheme definitions because it makes the proofs typographically cleaner and the theorems true.

Induction over things besides numbers

Persons other than pure mathematicians often write programs that manipulate objects other than natural numbers—for example, strings, lists, trees, arrays, hash tables, programs, airline schedules, and so on. So far, the examples of induction we have seen deal with induction over the natural numbers. How does this help with these other domains?

One answer is that we can do inductive proofs over natural numbers that correspond to the size of the

STRINGS objects under consideration. Suppose we want to prove that ∀s P(s) for the domain of strings. Then define a proposition on natural numbers as follows:

Q(n) is the property that every string s of length n satisfies P(s).

Then a proof that ∀n Q(n) by induction on n establishes that ∀s P(s).

Similarly, we can prove things about trees by induction on the depth of the tree, or about programs by induction on the number of symbols in the program. These inductions can become quite cumbersome and unnatural. Let’s suppose we had never heard of the natural numbers; could we still do anything with strings and trees and programs? It turns out that we can define very natural induction principles for these sorts of objects without mentioning numbers at all.

An induction principle for strings

Let’s write a recursive algorithm for reversing a string and show that it works correctly.

SYMBOLS First, we will need to say what strings are. The elements of a string are symbols drawn from a set of ALPHABET symbols called an alphabet, which is usually denoted Σ. For example, if Σ= {a,b}, then strings can consist

of sequences of as and bs. Σ^{∗} denotes the set of all possible strings on the alphabet Σ, and always includes the empty string, which is denoted λ. Every symbol of Σ is also a string of length 1. (Note: this property in particular distinguishes strings from lists; but in general reasoning about strings is quite similar to reasoning about lists.)

CONCATENATION The basic way to construct strings is by concatenation. If s_{1} and s_{2} are strings, then their concatenation is also a string and is written s_{1}s_{2} or s_{1} · s_{2} if punctuation is needed for clarity. Concatenation is defined as follows:

Axiom 3.3 (Concatenation):

∀s∈ Σ^{∗} λ · s = s · λ = s
∀a∈ Σ ∀s_{1},s_{2}∈ Σ^{∗} (a · s_{1}) · s_{2} = a · (s_{1}· s_{2})

Just as Peano did for the natural numbers, we now provide axioms concerning what strings are, then we state an induction principle that allows proofs for all strings. Strings satisfy the following axioms:

Axiom 3.4 (Strings):

The empty string is a string: λ ∈ Σ^{∗}

Joining any symbol to a string gives a string: ∀a∈ Σ ∀s∈ Σ^{∗} a · s ∈ Σ^{∗} Because these axioms do not strictly define strings, we need an induction principle to construct proofs over all strings:

Axiom 3.5 (String Induction):

For any property P,

if P(λ) and ∀a∈ Σ ∀s∈ Σ^{∗} (P(s) ⟹ P(a · s)),

then ∀s∈ Σ^{∗} P(s).

This is a simple instance of structural induction, where a set of axioms defines the way in which objects

STRUCTURAL INDUCTION

in a set are constructed and an induction principle uses the construction step repeatedly to cover the entire

CONSTRUCTOR domain. Here, “·” is the constructor for the domain of strings, just as “+1” is the constructor for the natural numbers.

Notice that numbers appear nowhere in these axioms. We can do proofs thinking only about the objects in question. Let’s define a function that reverses a string and prove that it works.

Axiom 3.6 (Reverse):

r(λ) = λ
∀a∈ Σ ∀s∈ Σ^{∗}r(a · s) = r(s) · a

We would like to say something like “for every string s, r(s) reverses it.” To make this a precise theorem, we’ll need some independent, non-recursive way to say what we mean by reversing! There are several ways to do this, of which the easiest is to take advantage of “dot dot dot” notation:

Theorem 3.7: ∀s∈ Σ^{∗}, let s= a_{1}a_{2}...a_{n}; then r(s)= a_{n}...a_{2}a_{1}

Proof: The proof is by induction over the strings on the alphabet Σ. Let P(s) be the proposition that if s= a_{1}a_{2}...a_{n}, then r(s)= a_{n}...a_{2}a_{1}.

• Base case: prove P(λ).

P(λ) is the proposition that r(λ)= λ, which is true by definition.

• Inductive step: prove P(s) ⟹ P(a · s) for all a∈ Σ, s∈ Σ^{∗}.

1. The inductive hypothesis states that, for some arbitrary string s, if s= a_{1}a_{2}...a_{n}, then r(s)= a_{n}...a_{2}a_{1}.

2. To prove: for every symbol a, r(a · s)= a_{n}...a_{2}a_{1}a.

3. By the axiom for reverse,

r(a · s) = r(s) · a by the reverse axiom
= a_{n}...a_{2}a_{1}a by the inductive hypothesis

Hence, by the string induction principle, for every string s, r(s) reverses it. □

We could alternatively have proven this theorem by induction over the length of the input string. It is an excellent exercise to work out the details of how to do this, and compare to the above method.

Induction over binary trees

Trees are a fundamental data structure in computer science, underlying efficient implementations in many areas including databases, graphics, compilers, editors, optimization, game-playing, and so on. Trees are

also used to represent expressions in formal languages. Here we study their most basic form: the binary

BINARY TREE tree. Binary trees include lists (as in Lisp and Scheme), which have nil as the rightmost leaf.

ATOMS In the theory of binary trees, we begin with atoms, which are trees with no branches. A is the set of atoms, which may or may not be finite. We construct trees (T) using the • (cons) operator. (In practice, any object can be an atom as long as it’s distinguishable as one.)

Axiom 3.7 (Binary Trees):

Every atom is a tree: ∀a∈ A [a ∈ T]

Consing any two trees gives a tree: ∀t_{1},t_{2}∈ T [t_{1}• t_{2} ∈ T]

The induction principle for trees says that if P holds for all atoms, and if the truth of P for any two trees implies the truth of P for their composition, then P holds for all trees:

Axiom 3.8 (Binary Tree Induction):

For any property P, if ∀a∈ A P(a)

and ∀t_{1},t_{2}∈ T [P(t_{1}) ∧ P(t_{2}) ⟹ P(t_{1}• t_{2})] then ∀t ∈ T P(t).

Many useful predicates and functions can be defined on trees, including

• leaf(a,t) is true iff atom a is a leaf of tree t.

• t_{1} ≺ t_{2} is true iff tree t_{1} is a proper subtree of tree t_{2}.

• count(t) denotes the number of leaves of the tree t.

DEPTH • depth(t) denotes the depth of the tree, where any atom has depth 0.

• balanced(t) is true iff t is a balanced binary tree.

Here we define leaf, leaving the others as exercises:

Axiom 3.9 (Leaf):

∀a∈ A ∀t ∈ T leaf(t,a) ⇔ t = a
∀a∈ A ∀t_{1},t_{2}∈ T leaf(a,t_{1}• t_{2}) ⇔ leaf(a,t_{1}) ∨ leaf(a,t_{2})

It’s not easy to prove that definitions of such basic functions are correct, since the “specification” of the function is hard to write in any form that is simpler than the definition itself. Let’s look at a slightly less simple function: the function maxleaf(t) returns the largest leaf of the tree t, where the atoms are constrained to be numbers.

Axiom 3.10 (Maxleaf):

∀a∈ A maxleaf(a) = a
∀t_{1},t_{2}∈ T maxleaf(t_{1}• t_{2}) = max(maxleaf(t_{1}),maxleaf(t_{2}))

The function maxleaf is “correct” if it satisfies two properties: first, maxleaf(t) has to be greater than or equal to every leaf of t; second (and often forgotten), maxleaf(t) has to be a leaf of t!

Let’s prove the second property first:

Theorem 3.8: For every tree, t, maxleaf(t) is a leaf of t.

Proof: The proof is by induction over the binary trees on the atoms A. Let P(t) be the proposition leaf(maxleaf(t),t).

• Base case: prove ∀a∈ A P(a).

P(a) is the proposition that leaf(maxleaf(a),a), which is equivalent by substitution to the proposition leaf(a,a), which is true by definition.

• Inductive step: prove P(t_{1}) ∧ P(t_{2}) ⟹ P(t_{1}• t_{2}) for all t_{1},t_{2}∈ T.

1. The inductive hypothesis states that leaf(maxleaf(t_{1}),t_{1}) ∧ leaf(maxleaf(t_{2}),t_{2}).

2. To prove: leaf(maxleaf(t_{1}• t_{2}),t_{1}• t_{2}).

3. By the definition above, maxleaf(t_{1}• t_{2}) = max(maxleaf(t_{1}),maxleaf(t_{2})).

4. Since ∀x,y [(max(x,y)= x) ∨ (max(x,y)= y)], we have (maxleaf(t_{1}• t_{2})= maxleaf(t_{1})) ∨ (maxleaf(t_{1}• t_{2})= maxleaf(t_{2})).

5. Substituting in the induction hypothesis, we obtain leaf(maxleaf(t_{1}• t_{2}),t_{1}) ∨ leaf(maxleaf(t_{1}• t_{2}),t_{2}).

6. Hence, by the definition of leaf, leaf(maxleaf(t_{1}• t_{2}),t_{1}• t_{2}).

Hence, by the binary induction principle, for every tree t, maxleaf(t) is a leaf of t. □

The other part of the verification is the following (the proof is left as an exercise):

Theorem 3.9: For every tree, t, maxleaf(t) is greater than or equal to every leaf of t.

Tree induction seems very natural. Could we do a similar proof using natural number induction? Certainly we can prove facts about trees by induction over the depth of the tree. P(n) would state that all trees of depth n satisfy some property Q. Unfortunately, the inductive step for a simple induction would look like this:

Given: all trees t of depth n satisfy Q(t) Prove: all trees t of depth n + 1 satisfy Q(t)

This is usually impossible: for a tree of depth n + 1, one subtree has depth n, but not necessarily the other. Strong induction over the depth of the tree does work; in fact it can always be used instead of tree induction.

Induction over pairs of natural numbers

Often we need to prove properties over the Cartesian product of some given sets. The Cartesian product

CARTESIAN PRODUCT

PAIRS of sets A and B is written A× B. It is the set of all pairs (a,b) where a∈ A and b∈ B. For example, the set N× N is the set of all pairs of natural numbers. Such sets arise when we prove properties of functions with two arguments, when we prove facts about all points on a grid, etc.

Let’s look at an example: the knight’s tour. We will prove that a knight starting at (0,0) can visit every square on the unbounded nonnegative quadrant. Figure 1 shows (part of) the infinite board and illustrates the moves a knight can make.

Figure 1: The knight’s tour, showing the “base case” squares, the possible legal moves for a knight, and the “inductive step.”

To prove this result, we’ll need some facts about knight’s moves. In particular, we’ll need the following:

Axiom 3.11 (Knight’s Move):

If square (x ± 1,y ± 2) or (x ± 2,y ± 1) is reachable by a knight, then square (x,y) is reachable by a knight.

We’ll also need an induction principle for pairs of natural numbers. The idea for the knight’s move proof is to establish a region that is reachable and then to show that any square adjacent to that region is reachable; hence the region grows to fill the unbounded quadrant. There are many ways to define the shape of this region; we’ll use the triangular region shown in Figure 1.

Our induction principle is, informally, that if the truth of P for every pair (x′,y′) in the region “just below” (x,y) implies the truth of P for (x,y), then P is true for all (x,y). Notice that this is a strong induction principle.

Axiom 3.12 (Strong Induction (Pairs)):

For any property P,

if ∀x,y∈ N

[∀x′,y′∈ N (x′+ y′) < (x + y) ⟹ P(x′,y′)] ⟹ P(x,y) then ∀x,y∈ N P(x,y).

But where is the base case? Actually, it’s there but hidden. When (x,y) = (0,0), the condition [∀x′,y′∈ N (x′+ y′) < (x + y) ⟹ P(x′,y′)] is vacuously true because there are no such pairs. Hence P(0,0) is part of the premise to be proved. More generally, the “base case” is the set of (x,y) pairs for which the inductive hypothesis does not suffice to provide a proof.

Now we are ready to prove our theorem:

Theorem 3.10: ∀x,y∈ N, the square (x,y) is reachable by a knight starting at (0,0).

Proof: The proof is by strong induction over the pairs of natural numbers. Let P(x,y) be the proposition that square (x,y) is reachable by a knight starting at (0,0).

• Base case: the propositions P(0,0), P(0,1), P(0,2), P(1,0), P(1,1), P(2,0), for which x+y ≤ 2, must be established separately. Each of these can be established by appropriate application of the knight’s move axiom.

• Inductive step: prove that, for all (x,y) such that x + y > 2, [∀x′,y′∈ N (x′+ y′) < (x + y) ⟹ P(x′,y′)] ⟹ P(x,y).

1. The inductive hypothesis states that, for all x′,y′∈ N such that (x′+ y′) < (x + y), the square (x′,y′) is reachable from (0,0).

2. All the squares (x′,y′)= (x −2,y±1) and (x′,y′)= (x ±1,y−2) satisfy the condition (x′+y′) < (x + y).

3. For any x,y∈ N such that x + y > 2, at least one of these squares is on the board, i.e., satisfies x′,y′∈ N (proof by cases).

4. Hence, by the knight’s move axiom, (x,y) is reachable from (0,0).

Hence, by the strong induction principle for pairs, every square in the unbounded positive quadrant is reachable by a knight from (0,0). □

The proof could also be done by strong induction on the natural numbers using n= x + y as the induction variable. Which is more elegant is perhaps a matter of taste; but the important insight is the use of a suitable notion of “smaller” on pairs of natural numbers. For some proofs, “smaller” can be defined as “at least one of the pair is smaller and the other is no bigger”, which gives rectangular regions that, stepwise, fill up the quadrant. In the knight’s tour problem, however, some of the required moves violate this ordering.

Well-founded induction

Looking at all the induction principles we have seen so far, one recurring theme stands out: from properties of “smaller” elements, we prove properties of a “larger” element. n is smaller than n + 1; s is smaller than a · s; t_{1} and t_{2} are smaller than t_{1}• t_{2}; and so on.

The strong induction principle for pairs, stated in the preceding section, gives a clue as to how to formalize this idea into a general induction principle. We simply supply a generalized notion of “smaller than” instead of using <. We denote this relation ≺, which is assumed to be defined on whatever set X we are interested in (natural numbers, sets, trees, pairs, strings, lists, airline schedules, etc.). For induction to work, we require that ≺ have the property of well-foundedness:

WELL-FOUNDED Definition 3.4 (Well-founded): A relation ≺ on X is well-founded if there can be no infinite decreasing sequences of elements of X related by ≺.

Given this, we can state the principle of well-founded induction, of which all our other principles are

WELL-FOUNDED INDUCTION

special cases:

Axiom 3.13 (Well-Founded Induction):

For any property P, and any wellfounded relation ≺ on X, if ∀x∈ X [[∀y∈ X y ≺ x ⟹ P(y)] ⟹ P(x)] then ∀x∈ X P(x).

As with induction over pairs, the well-founded induction principle includes the requirement for establishing the “base case”—that is, proving P(x) independently for all those x where the inductive hypothesis does not suffice.

The property of well-foundedness is easy to see for all the cases we have covered. There is also a generalized equivalent of well-ordering:

WELL-ORDERED Definition 3.5 (Well-ordering): A set X is well-ordered by the relation ≺ iff every nonempty subset of X has at least one minimal element with respect to ≺.

The following very general theorem can be proved:

Theorem 3.11: A relation ≺ on Xis well-founded iff Xis well-ordered by ≺.

Although this seems very abstract and useless, it is in fact used all the time by programmers who write recursive functions that do complex things to their arguments. Consider the following recursive skeleton:

f(x) = if B(x) then k else f(g(x))

This will terminate iff g(x) ≺ x for some well-ordering of X with minimal element(s) satisfying B(x). Thus, the programmer must be sure that repeated application of g cannot generate an infinite sequence of values that do not satisfy B.

Sometimes, “smaller” can be surprisingly nonobvious. Consider the following function on the natural numbers:

f(0) = 1; f(1) = 1
if n > 1 is even then f(n) = f(n/2), else f(n) = f(3n + 1).

The Collatz conjecture states that ∀n∈ N f(n) = 1. You may wish to check this out for various values of n.

COLLATZ CONJECTURE

No proof is known.