[FrontPage] [TitleIndex] [WordIndex

Note: You are looking at a static copy of the former PineWiki site, used for class notes by James Aspnes from 2003 to 2012. Many mathematical formulas are broken, and there are likely to be other bugs as well. These will most likely not be fixed. You may be able to find more up-to-date versions of some of these notes at http://www.cs.yale.edu/homes/aspnes/#classes.

PDF version

1. Simple induction

Most of the ProofTechniques we've talked about so far are only really useful for proving a property of a single object (although we can sometimes use generalization to show that the same property is true of all objects in some set if we weren't too picky about which single object we started with). Mathematical induction (which mathematicians just call induction) is a powerful technique for showing that some property is true for many objects, where you can use the fact that it is true for small objects as part of the proof that it is true for large objects.

The basic framework for induction is as follows: given a sequence of statements P(0), P(1), P(2), we'll prove that P(0) is true (the base case), and then prove that for all k, P(k) ⇒ P(k+1) (the induction step). We then conclude that P(n) is in fact true for all n.

1.1. Why induction works

There are three ways to show that induction works, depending on where you got your natural numbers from.

Peano axioms

If you start with the PeanoAxioms, induction is one of them. Nothing more needs to be said.

Well-ordering of the naturals

A set is well-ordered if every subset has a smallest element. (An example of a set that is not well-ordered is the integers ℤ.) If you build the natural numbers using 0 = { } and x+1 = x ∪ {x}, it is possible to prove that the resulting set is well-ordered. Because it is well-ordered, if P(n) does not hold for all n, there is a smallest n for which P(n) is false. But then either this n = 0, contradicting the base case, or P(n-1) is true (because otherwise n would not be the smallest) and P(n) is false, contradicting the induction step.

Method of infinite descent

The original version, due to Fermat, goes like this: Suppose P(n) is false for some n > 0. Since P(n-1) ⇒ P(n) is logically equivalent to ¬P(n) ⇒ ¬P(n-1), we can conclude (using the induction step) ¬P(n-1). Repeat until you reach 0. The problem with this version is that the "repeat" step is in effect using an induction argument. The modern solution to this problem is to recast the argument to look like the well-ordering argument above, by assuming that n is the smallest n for which P(n) is false and asserting a contradiction once you prove ¬P(n-1). Historical note: Fermat may have used this technique to construct a plausible but invalid proof of his famous "Last Theorem" that an+bn=cn has no non-trivial integer solutions for n > 2.

1.2. Examples

2. Strong induction

Sometimes when proving that the induction hypothesis holds for n+1, it helps to use the fact that it holds for all n' < n+1, not just for n. This sort of argument is called strong induction. Formally, it's equivalent to simple induction: the only difference is that instead of proving ∀k P(k) ⇒ P(k+1), we prove ∀k (∀m≤k Q(m)) ⇒ Q(k+1). But this is exactly the same thing if we let P(k) ≡ ∀m≤k Q(m), since if ∀m≤k Q(m) implies Q(k+1), it also implies ∀m≤k+1 Q(m), giving us the original induction formula ∀k P(k) ⇒ P(k+1).

2.1. Examples

3. Recursion

A definition with the structure of an inductive proof (give a base case and a rule for building bigger structures from smaller ones) Structures defined in this way are recursively-defined.

Examples of recursively-defined structures:

Finite Von Neumann ordinals
A finite von Neumann ordinal is either (a) the empty set ∅, or (b) x ∪ { x }, where x is a finite von Neumann ordinal.
Complete binary trees

A complete binary tree consists of either (a) a leaf node, or (b) an internal node (the root) with two complete binary trees as children (or subtrees).

Boolean formulas
A boolean formula consists of either (a) a variable, (b) the negation operator applied to a Boolean formula, (c) the AND of two Boolean formulas, or (d) the OR of two Boolean formulas. A monotone Boolean formula is defined similarly, except that negations are forbidden.
Finite sequences, recursive version

Before we defined a finite sequence as a function from some natural number (in its set form: n = { 0, 1, 2, ..., n-1 }) to some range. We could also define a finite sequence over some set S recursively, by the rule: [ ] (the empty sequence) is a finite sequence, and if a is a finite sequence and x∈S, then (x,a) is a finite sequence. (Fans of LISP will recognize this method immediately.)

Key point is that in each case the definition of an object is recursive---the object itself may appear as part of a larger object. Usually we assume that this recursion eventually bottoms out: there are some base cases (e.g. leaves of complete binary trees or variables in Boolean formulas) that do not lead to further recursion. If a definition doesn't bottom out in this way, the class of structures it describes might not be well-defined (i.e., we can't tell if some structure is an element of the class or not).

3.1. Recursively-defined functions

We can also define functions on recursive structures recursively:

The depth of a binary tree

For a leaf, 0. For a tree consisting of a root with two subtrees, 1+max(d1, d2), where d1 and d2 are the depths of the two subtrees.

The value of a Boolean formula given a particular variable assignment
For a variable, the value (true or false) assigned to that variable. For a negation, the negation of the value of its argument. For an AND or OR, the AND or OR of the values of its arguments. (This definition is not quite as trivial as it looks, but it's still pretty trivial.)

Or we can define ordinary functions recursively:

The Fibonacci series

Let F(0) = F(1) = 1. For n > 1, let F(n) = F(n-1) + F(n-2).


Let 0! = 1. For n > 0, let n! = n × (n-1)!.

3.2. Recursive definitions and induction

Recursive definitions have the same form as an induction proof. There are one or more base cases, and one or more recursion steps that correspond to the induction step in an induction proof. The connection is not surprising if you think of a definition of some class of objects as a predicate that identifies members of the class: a recursive definition is just a formula for writing induction proofs that say that certain objects are members.

Recursively-defined objects and functions also lend themselves easily to induction proofs about their properties; on general structures, such induction arguments go by the name of structural induction.

4. Structural induction

For finite structures, we can do induction over the structure. Formally we can think of this as doing induction on the size of the structure or part of the structure we are looking at.


Every complete binary tree with n leaves has n-1 internal nodes

Base case is a tree consisting of just a leaf; here n = 1 and there are n - 1 = 0 internal nodes. The induction step considers a tree consisting of a root and two subtrees. Let n1 and n2 be the number of leaves in the two subtrees; we have n1+n2 = n; and the number of internal nodes, counting the nodes in the two subtrees plus one more for the root, is (n1-1)+(n2-1)+1 = n1+n2 - 1 = n-1.

Monotone Boolean formulas generate monotone functions
What this means is that changing a variable from false to true can never change the value of the formula from true to false. Proof is by induction on the structure of the formula: for a naked variable, it's immediate. For an AND or OR, observe that changing a variable from false to true can only leave the values of the arguments unchanged, or change one or both from false to true (induction hypothesis); the rest follows by staring carefully at the truth table for AND or OR.
Bounding the size of a binary tree with depth d

We'll show that it has at most 2d+1-1 nodes. Base case: the tree consists of one leaf, d = 0, and there are 20+1-1 = 2-1 = 1 nodes. Induction step: Given a tree of depth d > 1, it consists of a root (1 node), plus two subtrees of depth at most d-1. The two subtrees each have at most 2d-1+1-1 = 2d-1 nodes (induction hypothesis), so the total number of nodes is at most 2(2d-1)+1 = 2d+1+2-1 = 2d+1-1.


2014-06-17 11:58