CS 3133, A Term 1998, Professor Alvarez
Mathematical Background Material
This page summarizes some of the basic mathematical concepts and tools
that are used repeatedly in CS 3133 as well as in any treatment of the
foundations (theory) of computation. The descriptions contained here are
for the most part brief reviews with pointers to further study and are in
no way intended to be exhaustive. It is assumed that you have already
mastered the content of a typical first course in discrete mathematics.
I encourage you to do the exercises given below and hope that this
material will help you get the most out of this course. Have fun!
Sets, relations, and functions
For our purposes, a set is merely a collection of objects in the
everyday sense. The objects of the collection are called the members
or elements of the set. A set is completely determined by its members,
so that saying that two sets are equal is the same as saying that the two sets
have the same members. This definition can be made more formal, and issues that
are interesting (or silly, depending on your perspective) arise almost
immediately. For example, consider the set S whose members are all sets that
are not members of themselves; then is S a member of S, or not?
(see e.g. the book "Naive Set Theory" by Paul Halmos).
The usual braces will be used to define sets.
Thus, {Tom, Mary, Paul} refers to the set whose elements are Tom, Mary, and
Paul, while {x | x is a unicorn} refers to the set of all unicorns.
The vertical bar here is short for "such that".
Notice that there is no need for a set to have any elements at all:
there is a unique empty set that has no elements
(suppose there are two empty sets A and B.
Then A and B must have the same elements, because there aren't
any "witnesses" to the contrary).
The symbol N denotes the set of natural numbers,
which are the "counting numbers" 0, 1, 2, 3, 4, ...
There isn't a uniform agreement regarding whether 0 is a
natural number or not. This shouldn't be surprising since,
for example, the "first floor" of a building is sometimes the
second one up from ground level, especially in Europe.
Set-building operations
Given sets A and B, one can construct new sets as follows:
- The union of A and B is the set A U B containing all elements of A,
all elements of B, and no other elements.
- The intersection of A and B is the set A ^ B whose elements are the
objects that are simultaneously elements of both A and B.
- The difference of A and B is the set A-B whose elements are those
elements of A that are not elements of B.
- The complement of A is the set A' of all objects belonging to
some predetermined universal set that depends on the context, that
are not elements of A. So A' is really just U-A, where U is the universal set.
- The Cartesian product of A and B is the set A x B whose elements
are the pairs (a,b), where a ranges over all elements of A and b ranges over
all elements of B.
One has the following duality relations (sometimes called
DeMorgan's laws) for the union and intersection operations:
- (A U B)' = A' ^ B'
- (A ^ B)' = A' U B'
Exercise.
Give an argument to convince a skeptical but reasonable jury that
DeMorgan's laws (above) are true. Since the jury is skeptical,
it's not enough to say, for example, "they're true because I
say so", or, "I tried them out with these particular sets A
and B that I'm bringing into evidence and it worked fine",
because the jury will suspect that the result could be different
for other choices of A and B. So your argument has to be rational
and general. We're looking for "proof beyond a shadow of a doubt"
here. On the other hand, the jury is reasonable, so once there is
no way for your argument to fail, they will accept it.
Hint: you wish to show in each case that two sets C and D are the same;
thus, you must show that C and D have the same elements.
To do this, first show that every element of C is also an element of D.
Then show that every element of D is also an element of C.
Functions
A function f: B -> A from a set B to another set A is a "black box"
the inner workings of which may be unknown, but which somehow assigns a
well-defined element f(b) of A to each element b of B.
A function f: B -> A maps B onto A if no matter what element
a of A you pick, there will be some element b of B such that f(b)=a, i.e.
such that b is mapped to a by the function f.
A function f: B -> A is said to be one-to-one if different elements
of B are mapped to different elements of A, or, equivalently, if whenever
b1 and b2 are elements of B such that
f(b1) = f(b2), then b1 = b2.
Given functions g: C -> B and f: B -> A, one may form a new function
f * g: C -> A called the composition of g and f, defined by:
(f*g)(c) = f( g(c) ) for all elements c of C.
The set of all functions f: B -> A from B into A is denoted AB.
Exercises.
- Consider the function f: N -> N defined by f(n) = n2.
Is this function one-to-one? Is it onto? Explain your answers with arguments that
would convince a skeptical but reasonable jury.
- Assume given functions g: C -> B and f: B -> A.
- Show that if g and f are both onto,
then the composite function f * g: C -> A is also onto.
Give an example to show that if f is onto but g is not,
then the conclusion may fail to be true.
- Show that even if the composite function f * g is one-to-one,
the function f need not be. What additional condition on g will
force f to be one-to-one in this situation?
Countable sets
You can count the elements of lots of sets using natural numbers.
This includes most of the sets encountered in CS 3133.
Counting the elements of a set A means finding a function
f: N -> A that maps onto A. Since the counting
function f must be onto, it is necessary that no matter what
element a of A you pick, there will be some element n of N
that is mapped to a by f, i.e. f(n)=a.
This is a complicated way of saying that f determines an ordering
of all the elements of A, so that every element a of A is the n-th
element in this ordering for an appropriate value of n (i.e. a = f(n)).
Any set for which there is a counting function is said to be a
countable set.
A counting function f:N -> A is also called an enumeration
of the set A.
Exercise.
Show that the following sets are countable by explicitly
finding a counting function for each of them.
- The set {Tom, Mary, Paul}.
- The set E of all even natural numbers.
- The set N x N of all pairs (m,n) of natural numbers.
Hint: draw this set with m's as rows and n's as columns and then try to
see a way to erase the drawing in a systematic way.
Recursion and induction
Recursion is a method of defining countable sets, relations,
or functions in a step-by-step fashion. In the simplest context,
one wishes to define a function on the set of natural numbers.
Using recursion, one proceeds as follows:
- Define the "first value" f(0) of the function.
- Assuming that the values f(0) ... f(n) have been defined for some
value of n, define the "next value" f(n+1), possibly in terms of one
or more of the "previous values" f(0) ... f(n).
The toppling dominoes metaphor is again appropriate.
A number n topples when the corresponding function value f(n) is defined.
The basis step makes 0 topple, and the induction step topples n+1
once 0..n have toppled.
More generally, a definition by recursion constructs a set A in three steps:
- Basis step:
Define a starting set A0.
- Recursion step:
Assume that sets A0, ... An have been defined.
Then define the "next" set An+1 in terms of
A0, ... An.
- Closure:
Define the final set A as the union of all the sets An.
A set A constructed by recursion according to the above procedure
is called an inductive set.
The prototypical inductive set is the set N of natural numbers.
Induction is a method of reasoning which may be used
to show that every element of an inductive set has a certain property.
For example,
suppose that you want to find a formula for the sum Sn
of the first n natural numbers: Sn = 1 + 2 + 3 + ... + n.
You try some small values of n and find: S0=0, S1=1,
S2=3, S3=6, S4=10.
After fiddling with these values for a while you see that
the formula Sn=n(n+1)/2 works for these values of n.
But what about larger values? Since there are infinitely
many values of n to check, in order to reach some conclusion
about the validity of the formula before you die you have to come up
with some abbreviated logical argument that applies to all values of n
without actually checking them one by one.
Here's how induction does the trick:
- (Basis step) Check the first value of n: n=0.
Yes, S0=0=0(0+1)/2.
- (Induction step) Assume that you've checked all the values up to
and including some n, and show that the next value, n+1, also checks:
Sn+1
= Sn + (n+1)
= n(n+1)/2 + (n+1)
= (n+1)(n/2 + 1) = (n+1)(n+2)/2.
Yes!
So now you see that the natural numbers are lined up like dominoes
and the above two steps make all of them fall. The basis step makes
the first one fall, and the induction step shows that if all the dominoes
up to a certain point have fallen then the next one in line falls too.
This implies that all of them fall, i.e. in this case that the formula
works for all values of n.
More generally, if one wishes to prove that a statement A(n) is true
for every n, an induction proof would proceed as follows.
- Basis: one proves that A(0) is true.
- Induction step: one assumes (induction hypothesis)
that A(n) is true, and proves that this hypothesis implies that
A(n+1) is true.
Exercises.
- By trial and error, guess a simple formula in terms of n
for the sum Sn = 1 + 4 + 9 + ... + n2
of the squares of the first n positive integers.
Then prove your formula by induction.
- Use induction to prove that 2n > n
for all natural numbers n (an unrelated proof appears in an exercise
on 0/1 sequences in the section on diagonalization below).
Uncountable sets, diagonalization
There are some sets that simply cannot be counted. They just
have too many elements! This was first understood by Cantor in
the 19th century.
I'll give an example of Cantor's famous diagonalization argument,
which shows that certain sets are not countable.
See Halmos' book for more details.
Remember Paul Revere's ride? Here is an excerpt from the poem
by Henry Wadsworth Longfellow:
He said to his friend, "If the British march
By land or sea from the town to-night,
Hang a lantern aloft in the belfry arch
Of the North Church tower as a signal light,--
One if by land, and two if by sea;
And I on the opposite shore will be,
Ready to ride and spread the alarm
Through every Middlesex village and farm,
For the country folk to be up and to arm."
The infinitely many lamps encoding
The one lamp / two lamps encoding used by Revere and his friend was fine
for the two relevant alternatives of land and sea.
But what if you wanted to be able to represent infinitely many alternatives?
You can imagine infinitely
many towers, each with a lamp that can be either on or off.
Being on is like displaying two lamps in the Paul Revere story,
and being off is like displaying just one lamp. With the
infinitely many towers being visible at once, you can
convey infinitely many messages, one for each combination
of "on" and "off" (e.g. odd-numbered lamps may be on
and even-numbered lamps may be off, or the only lamps
that are on may be the powers of 2, or whatever other
combination you like). What Cantor showed is that the
set of all possible messages here is so large that
it is not countable! In other words, there is no
counting function that maps the set N of
natural numbers onto the set of all messages,
and in fact any would-be counting function will
necessarily "miss" infinitely many messages.
Impossibility and proof by contradiction
Take a moment to think about what is being claimed here.
In contrast to simply exhibiting a counting function that
works, as in proving that a set is countable, in proving
that a set is uncountable one is required to show that no
matter how clever a construction of a counting function may be,
it cannot work.
In order to convince a jury of this, you must give an
argument that does not depend on any details of the
would-be counting function, so that it will apply to
all possible functions that could ever be concocted.
Cantor gave a proof by contradiction.
That is, he began by assuming that, contrary to the
intended conclusion, there actually is a
counting function for the set of messages, and
then he proceeded to show that this assumption alone
leads to a contradiction, regardless of any details
of the particular counting function being considered.
A contradiction means that some statement must
simultaneously be true and false, or, equivalently,
that two mutually exclusive alternatives must occur
simultaneously.
A contradiction, by definition, can't happen!
Since it is the initial assumption that there
is a counting function which leads to a contradiction,
we conclude that this assumption must be false, and
thus that there can be no counting function at all
for the set of messages.
Cantor's diagonalization argument
With the above plan in mind, let M denote the set
of all possible messages in the infinitely many lamps
encoding, and assume that there is a function f: N -> M
that maps onto M. We want to show that this assumption
leads to a contradiction. Here goes. The would-be counting
function f supposedly enumerates all of the elements of M.
f(0) is the first message, then f(1), then f(2), and so on.
Picture each message as a row containing the infinitely many
towers, each with its lamp turned either on or off.
f(0) is the top row, f(1) is the row below it, and in
general f(n+1) is the row just beneath f(n).
So you should be picturing infinitely many rows,
each containing infinitely many columns of towers.
Each row is one of the possible messages.
All rows supposedly exhaust the set of all possible messages.
Cantor's argument shows that there is a message that we missed!
He constructs this message by scanning down the "diagonal"
of the picture, looking first at the first tower of the
first row, then the second tower of the second row,
and in general the n-th tower of the n-th row.
If the lamp in the n-th tower of the n-th row
is on, turn it off. If the lamp is off, then
turn it on. Record the results to get a new message.
In other words, the lamp in the first tower of the
new message will be on if the lamp in the first tower of
the first row is off, and it will be off otherwise,
and in general the lamp in the n-th tower of the
new message will be on only if the lamp in the
n-th tower of the n-th row is off, etc.
The message thus constructed is not the same
as any of the rows in the would-be enumeration.
Why? Well, choose any row, say the n-th. Then
the new message differs from this n-th row
because their n-th towers differ: if the
n-th tower of the n-th row is on, then the n-th
tower of the new message is off, and vice-versa.
This shows that the new message wasn't listed
as part of the enumeration, which contradicts
the very notion of enumeration. Since contradictions
are impossible, we conclude that our initial
assumption that there is an enumeration must be
false. This means that the set of all messages
is not countable!
Exercise.
Use the basic idea behind Cantor's diagonalization argument
to show that there are more than n sequences of length n
consisting of 1's and 0's.
Hint: with the aim of obtaining a contradiction, begin by
assuming that there are n or fewer such sequences; list
these sequences as rows and then use diagonalization to
generate a new sequence that differs from all of the
original rows.
Rates of growth
Let f and g be functions from the set N of natural numbers to itself.
One writes f(n) = O(g(n)) if there exist numbers C and n0 such that
|f(n)| <= C |g(n)| for all n >= n0.
Examples.
- a n^2 + b^n + c = O(n^2)
I'll assume that a > 0. The other cases are left as an exercise.
First factor the left-hand side as
a n^2 (1 + b/(an) + c/(an^2)).
Now choose n0 so that
- b/(an0) + c/(an^20) <= 1
(this can be done because the expression on the left approaches 0
as n approaches infinity).
Then whenever n >= n0 we have also:
- b/(an) + c/(an^2) <= 1,
and it follows that whenever n >= n0 one has
- a n^2 + b^n + c <= 2a n^2
This proves the desired conclusion, for C=2a.
- n^2 = O(b^n) for any b>0
This requires calculus. Consider the quotient
- q(n) = n^2 / b^n,
where now n represents a positive real variable
rather than a positive integer value.
Both the numerator and the denominator of q(n) approach infinity
as n approaches infinity.
Applying L'Hopital's rule twice, one obtains:
- lim n -> infinity q(n)
= lim n -> infinity 2n / ((log b) b^n )
- = lim n -> infinity 2 / ((log b)^2 b^n ) = 0
This means that, in particular, there exists a value n0 such that
- n^2 / b^n <= 1 for all n >= n0
In other words, whenever n>=n0, one has:
- n^2 <= b^n
This implies that n^2 = O(b^n) as claimed.
- Generalizing the preceding example:
- if lim n -> infinity f(n) / g(n) = 0,
then f(n) = O(g(n)).
- If f(n) = O(g(n)) and h(n) = O(k(n)), then
- f(n) + h(n) = O(|g(n)| + |k(n)|)
- f(n) h(n) = O(g(n) k(n))
