CS 3133, A Term 1998, Professor Alvarez
Mathematical Background Material

This page summarizes some of the basic mathematical concepts and tools that are used repeatedly in CS 3133 as well as in any treatment of the foundations (theory) of computation. The descriptions contained here are for the most part brief reviews with pointers to further study and are in no way intended to be exhaustive. It is assumed that you have already mastered the content of a typical first course in discrete mathematics. I encourage you to do the exercises given below and hope that this material will help you get the most out of this course. Have fun!

Sets, relations, and functions

For our purposes, a set is merely a collection of objects in the everyday sense. The objects of the collection are called the members or elements of the set. A set is completely determined by its members, so that saying that two sets are equal is the same as saying that the two sets have the same members. This definition can be made more formal, and issues that are interesting (or silly, depending on your perspective) arise almost immediately. For example, consider the set S whose members are all sets that are not members of themselves; then is S a member of S, or not? (see e.g. the book "Naive Set Theory" by Paul Halmos).

The usual braces will be used to define sets. Thus, {Tom, Mary, Paul} refers to the set whose elements are Tom, Mary, and Paul, while {x | x is a unicorn} refers to the set of all unicorns. The vertical bar here is short for "such that". Notice that there is no need for a set to have any elements at all: there is a unique empty set that has no elements (suppose there are two empty sets A and B. Then A and B must have the same elements, because there aren't any "witnesses" to the contrary).

The symbol N denotes the set of natural numbers, which are the "counting numbers" 0, 1, 2, 3, 4, ... There isn't a uniform agreement regarding whether 0 is a natural number or not. This shouldn't be surprising since, for example, the "first floor" of a building is sometimes the second one up from ground level, especially in Europe.

Set-building operations

Given sets A and B, one can construct new sets as follows:
  1. The union of A and B is the set A U B containing all elements of A, all elements of B, and no other elements.
  2. The intersection of A and B is the set A ^ B whose elements are the objects that are simultaneously elements of both A and B.
  3. The difference of A and B is the set A-B whose elements are those elements of A that are not elements of B.
  4. The complement of A is the set A' of all objects belonging to some predetermined universal set that depends on the context, that are not elements of A. So A' is really just U-A, where U is the universal set.
  5. The Cartesian product of A and B is the set A x B whose elements are the pairs (a,b), where a ranges over all elements of A and b ranges over all elements of B.
One has the following duality relations (sometimes called DeMorgan's laws) for the union and intersection operations:

Exercise. Give an argument to convince a skeptical but reasonable jury that DeMorgan's laws (above) are true. Since the jury is skeptical, it's not enough to say, for example, "they're true because I say so", or, "I tried them out with these particular sets A and B that I'm bringing into evidence and it worked fine", because the jury will suspect that the result could be different for other choices of A and B. So your argument has to be rational and general. We're looking for "proof beyond a shadow of a doubt" here. On the other hand, the jury is reasonable, so once there is no way for your argument to fail, they will accept it.
Hint: you wish to show in each case that two sets C and D are the same; thus, you must show that C and D have the same elements. To do this, first show that every element of C is also an element of D. Then show that every element of D is also an element of C.


A function f: B -> A from a set B to another set A is a "black box" the inner workings of which may be unknown, but which somehow assigns a well-defined element f(b) of A to each element b of B. A function f: B -> A maps B onto A if no matter what element a of A you pick, there will be some element b of B such that f(b)=a, i.e. such that b is mapped to a by the function f. A function f: B -> A is said to be one-to-one if different elements of B are mapped to different elements of A, or, equivalently, if whenever b1 and b2 are elements of B such that f(b1) = f(b2), then b1 = b2.

Given functions g: C -> B and f: B -> A, one may form a new function f * g: C -> A called the composition of g and f, defined by: (f*g)(c) = f( g(c) ) for all elements c of C.

The set of all functions f: B -> A from B into A is denoted AB.

  1. Consider the function f: N -> N defined by f(n) = n2. Is this function one-to-one? Is it onto? Explain your answers with arguments that would convince a skeptical but reasonable jury.
  2. Assume given functions g: C -> B and f: B -> A.
    1. Show that if g and f are both onto, then the composite function f * g: C -> A is also onto. Give an example to show that if f is onto but g is not, then the conclusion may fail to be true.
    2. Show that even if the composite function f * g is one-to-one, the function f need not be. What additional condition on g will force f to be one-to-one in this situation?

Countable sets

You can count the elements of lots of sets using natural numbers. This includes most of the sets encountered in CS 3133. Counting the elements of a set A means finding a function f: N -> A that maps onto A. Since the counting function f must be onto, it is necessary that no matter what element a of A you pick, there will be some element n of N that is mapped to a by f, i.e. f(n)=a. This is a complicated way of saying that f determines an ordering of all the elements of A, so that every element a of A is the n-th element in this ordering for an appropriate value of n (i.e. a = f(n)). Any set for which there is a counting function is said to be a countable set. A counting function f:N -> A is also called an enumeration of the set A.

Exercise. Show that the following sets are countable by explicitly finding a counting function for each of them.
  1. The set {Tom, Mary, Paul}.
  2. The set E of all even natural numbers.
  3. The set N x N of all pairs (m,n) of natural numbers.
    Hint: draw this set with m's as rows and n's as columns and then try to see a way to erase the drawing in a systematic way.

Recursion and induction

Recursion is a method of defining countable sets, relations, or functions in a step-by-step fashion. In the simplest context, one wishes to define a function on the set of natural numbers. Using recursion, one proceeds as follows:
  1. Define the "first value" f(0) of the function.
  2. Assuming that the values f(0) ... f(n) have been defined for some value of n, define the "next value" f(n+1), possibly in terms of one or more of the "previous values" f(0) ... f(n).
The toppling dominoes metaphor is again appropriate. A number n topples when the corresponding function value f(n) is defined. The basis step makes 0 topple, and the induction step topples n+1 once 0..n have toppled.

More generally, a definition by recursion constructs a set A in three steps:

  1. Basis step: Define a starting set A0.
  2. Recursion step: Assume that sets A0, ... An have been defined. Then define the "next" set An+1 in terms of A0, ... An.
  3. Closure: Define the final set A as the union of all the sets An.
A set A constructed by recursion according to the above procedure is called an inductive set. The prototypical inductive set is the set N of natural numbers.

Induction is a method of reasoning which may be used to show that every element of an inductive set has a certain property. For example, suppose that you want to find a formula for the sum Sn of the first n natural numbers: Sn = 1 + 2 + 3 + ... + n. You try some small values of n and find: S0=0, S1=1, S2=3, S3=6, S4=10. After fiddling with these values for a while you see that the formula Sn=n(n+1)/2 works for these values of n. But what about larger values? Since there are infinitely many values of n to check, in order to reach some conclusion about the validity of the formula before you die you have to come up with some abbreviated logical argument that applies to all values of n without actually checking them one by one.

Here's how induction does the trick:

  1. (Basis step) Check the first value of n: n=0. Yes, S0=0=0(0+1)/2.
  2. (Induction step) Assume that you've checked all the values up to and including some n, and show that the next value, n+1, also checks:
    Sn+1 = Sn + (n+1) = n(n+1)/2 + (n+1) = (n+1)(n/2 + 1) = (n+1)(n+2)/2. Yes!
So now you see that the natural numbers are lined up like dominoes and the above two steps make all of them fall. The basis step makes the first one fall, and the induction step shows that if all the dominoes up to a certain point have fallen then the next one in line falls too. This implies that all of them fall, i.e. in this case that the formula works for all values of n.

More generally, if one wishes to prove that a statement A(n) is true for every n, an induction proof would proceed as follows.

  1. Basis: one proves that A(0) is true.
  2. Induction step: one assumes (induction hypothesis) that A(n) is true, and proves that this hypothesis implies that A(n+1) is true.

  1. By trial and error, guess a simple formula in terms of n for the sum Sn = 1 + 4 + 9 + ... + n2 of the squares of the first n positive integers. Then prove your formula by induction.
  2. Use induction to prove that 2n > n for all natural numbers n (an unrelated proof appears in an exercise on 0/1 sequences in the section on diagonalization below).

Uncountable sets, diagonalization

There are some sets that simply cannot be counted. They just have too many elements! This was first understood by Cantor in the 19th century. I'll give an example of Cantor's famous diagonalization argument, which shows that certain sets are not countable. See Halmos' book for more details.

Remember Paul Revere's ride? Here is an excerpt from the poem by Henry Wadsworth Longfellow:

He said to his friend, "If the British march
By land or sea from the town to-night,
Hang a lantern aloft in the belfry arch
Of the North Church tower as a signal light,--
One if by land, and two if by sea;
And I on the opposite shore will be,
Ready to ride and spread the alarm
Through every Middlesex village and farm,
For the country folk to be up and to arm."

The infinitely many lamps encoding

The one lamp / two lamps encoding used by Revere and his friend was fine for the two relevant alternatives of land and sea. But what if you wanted to be able to represent infinitely many alternatives? You can imagine infinitely many towers, each with a lamp that can be either on or off. Being on is like displaying two lamps in the Paul Revere story, and being off is like displaying just one lamp. With the infinitely many towers being visible at once, you can convey infinitely many messages, one for each combination of "on" and "off" (e.g. odd-numbered lamps may be on and even-numbered lamps may be off, or the only lamps that are on may be the powers of 2, or whatever other combination you like). What Cantor showed is that the set of all possible messages here is so large that it is not countable! In other words, there is no counting function that maps the set N of natural numbers onto the set of all messages, and in fact any would-be counting function will necessarily "miss" infinitely many messages.

Impossibility and proof by contradiction

Take a moment to think about what is being claimed here. In contrast to simply exhibiting a counting function that works, as in proving that a set is countable, in proving that a set is uncountable one is required to show that no matter how clever a construction of a counting function may be, it cannot work. In order to convince a jury of this, you must give an argument that does not depend on any details of the would-be counting function, so that it will apply to all possible functions that could ever be concocted. Cantor gave a proof by contradiction. That is, he began by assuming that, contrary to the intended conclusion, there actually is a counting function for the set of messages, and then he proceeded to show that this assumption alone leads to a contradiction, regardless of any details of the particular counting function being considered. A contradiction means that some statement must simultaneously be true and false, or, equivalently, that two mutually exclusive alternatives must occur simultaneously. A contradiction, by definition, can't happen! Since it is the initial assumption that there is a counting function which leads to a contradiction, we conclude that this assumption must be false, and thus that there can be no counting function at all for the set of messages.

Cantor's diagonalization argument

With the above plan in mind, let M denote the set of all possible messages in the infinitely many lamps encoding, and assume that there is a function f: N -> M that maps onto M. We want to show that this assumption leads to a contradiction. Here goes. The would-be counting function f supposedly enumerates all of the elements of M. f(0) is the first message, then f(1), then f(2), and so on. Picture each message as a row containing the infinitely many towers, each with its lamp turned either on or off. f(0) is the top row, f(1) is the row below it, and in general f(n+1) is the row just beneath f(n). So you should be picturing infinitely many rows, each containing infinitely many columns of towers. Each row is one of the possible messages. All rows supposedly exhaust the set of all possible messages. Cantor's argument shows that there is a message that we missed! He constructs this message by scanning down the "diagonal" of the picture, looking first at the first tower of the first row, then the second tower of the second row, and in general the n-th tower of the n-th row. If the lamp in the n-th tower of the n-th row is on, turn it off. If the lamp is off, then turn it on. Record the results to get a new message. In other words, the lamp in the first tower of the new message will be on if the lamp in the first tower of the first row is off, and it will be off otherwise, and in general the lamp in the n-th tower of the new message will be on only if the lamp in the n-th tower of the n-th row is off, etc. The message thus constructed is not the same as any of the rows in the would-be enumeration. Why? Well, choose any row, say the n-th. Then the new message differs from this n-th row because their n-th towers differ: if the n-th tower of the n-th row is on, then the n-th tower of the new message is off, and vice-versa. This shows that the new message wasn't listed as part of the enumeration, which contradicts the very notion of enumeration. Since contradictions are impossible, we conclude that our initial assumption that there is an enumeration must be false. This means that the set of all messages is not countable!

Exercise. Use the basic idea behind Cantor's diagonalization argument to show that there are more than n sequences of length n consisting of 1's and 0's.
Hint: with the aim of obtaining a contradiction, begin by assuming that there are n or fewer such sequences; list these sequences as rows and then use diagonalization to generate a new sequence that differs from all of the original rows.

Rates of growth

Let f and g be functions from the set N of natural numbers to itself. One writes f(n) = O(g(n)) if there exist numbers C and n0 such that
|f(n)| <= C |g(n)|   for all n >= n0.


  1. a n^2 + b^n + c = O(n^2)
    I'll assume that a > 0. The other cases are left as an exercise. First factor the left-hand side as a n^2 (1 + b/(an) + c/(an^2)).
    Now choose n0 so that
    b/(an0) + c/(an^20) <= 1
    (this can be done because the expression on the left approaches 0 as n approaches infinity). Then whenever n >= n0 we have also:
    b/(an) + c/(an^2) <= 1,
    and it follows that whenever n >= n0 one has
    a n^2 + b^n + c <= 2a n^2
    This proves the desired conclusion, for C=2a.

  2. n^2 = O(b^n) for any b>0
    This requires calculus. Consider the quotient
    q(n) = n^2 / b^n,
    where now n represents a positive real variable rather than a positive integer value. Both the numerator and the denominator of q(n) approach infinity as n approaches infinity. Applying L'Hopital's rule twice, one obtains:
    lim n -> infinity q(n) = lim n -> infinity 2n / ((log b) b^n )
    = lim n -> infinity 2 / ((log b)^2 b^n ) = 0
    This means that, in particular, there exists a value n0 such that
    n^2 / b^n <= 1 for all n >= n0
    In other words, whenever n>=n0, one has:
    n^2 <= b^n
    This implies that n^2 = O(b^n) as claimed.

  3. Generalizing the preceding example:
    if lim n -> infinity f(n) / g(n) = 0, then f(n) = O(g(n)).

  4. If f(n) = O(g(n)) and h(n) = O(k(n)), then
    • f(n) + h(n) = O(|g(n)| + |k(n)|)
    • f(n) h(n) = O(g(n) k(n))