CS 2223 Mar 12 2018

Lecture Path: 01
Next

Expected reading: Today is the only day that you will have no expected readings
Expected demonstration: BINARY ARRAY SEARCH
Daily exercises: Closed Form, Anagrams, Defective Search
Classical selection: Haydn Symphony No 1 D major (1759)
Musical Selection: Bohemian Rhapsody, Queen (1975)

For the remainder of this course, I expect you to have completed all readings and demonstrations prior to the start of the lecture on which they are listed.

Before presenting this lecture, I will review the course structure.

Diamonds are found only in the dark places of the earth, truths are found only in the depths of thought.
Victor Hugo
Les Miserables

1 A Journey Of A Thousand Miles Begins With A Single Step

A perfect algorithm is like an artistic study in miniature. Consider the amazing portrait of Ginevra de’ Benci by Leonardo da Vinci [1474/1478]

Leonardo da Vinci painted this portrait – only 15x15 inches in size – when he was just 21 years old. It signaled a revolution in portraiture, fundamentally changing the way artists approached the subject. There is nothing extra in this artwork. Every brushstroke is meaningful and I encourage you to "zoom in" on the portrait. Click on the link below the image and you will be brought to the National Gallery of Art web site where you can click on the image there to explore. Right-click to see the mouse commands; you can double-click to zoom in and then move around to see the incredible detail of the picture.

Now let’s look at another elegant miniature, this time the BINARY ARRAY SEARCH algorithm which determine whether a sorted collection contains a target item. Here is a description from Jon Bentley’s amazing book, Programming Pearls:

BINARY [ARRAY] SEARCH solves the problem [of searching within a pre-sorted array] by keeping track of a range within the array in which T [i.e. the sought value] must be if it is anywhere in the array. Initially, the range is the entire array. The range is shrunk by comparing its middle element to T and discarding half the range. The process continues until T is discovered in the array, or until the range in which it must lie is known to be empty. In an N-element table, the search uses roughly log2(N) comparisons.

This algorithm, so briefly stated, is the foundation of so many efficient algorithms. Easily stated, but not so easily implemented. Bentley reports that given this description, 90% of professional programmers are unable to code this algorithm given several hours. What follows is a Java implementation:

Given the above program, I have some simple questions to ask:

Important questions are hilighted in orange. Pay attention to these!

• Is the while loop guaranteed to terminate?

• How many times will the while loop execute?

• Can you prove this code is correct?

By the end of this course, I want all students to appreciate the elegance and beauty of this algorithm. Its simplicity is legendary. And just as important, one needs correct implementations that are validated to handle all cases, especially the boundary cases that make it hard to translate algorithms into code.

There are some subtle features of the above code that you might not notice at first glance:

• Look at the order of the if statements. In the code above it first checks less than (lt), then greater than (gt) before concluding the values are equal (eq). Does it matter what order these checks are made? It sure does, as you can see below.

• Does this midpoint computation have any unexpected pitfalls?

  Look at the computation for the midpoint mid. This uses a standard integer truncation division operator. If you replace with a right-shift bitwise operator >> 1 the performance increases noticeably

The following table reports the result of an experiment. We generate a sorted array of 65,536 unique integers drawn from the range [-16777216, 16777216]. We then create a new list of 65,536 target integers in the range [-33554432, 33554432]. The target range is wider since we want to ensure we can search for numbers that are both too low and too high.

Any thoughts on the probability that a target search actually finds a value from the original sorted array? That is, of the T=33,554,432 individual searches, how many times will a random target value be successfully found in the original sorted array? Can you come up with a formula that estimates this number based on the definition of T, the initial sorted array, and the random target values?

We then want to generate a single trial run. Because computers are so fast these days, in each run we carry out the following task 512 times: search for each of the 65,536 target values in order. Thus there will be a total of 512 x 65,536 = 33,554,432 invocations of the contains method. All reported results are in seconds.

Program	PC	cs	ccc	PC/JS	iPhone/JS
eq_lt_gt_div	1.763	3.296	2.001	393.725	757.193
trial-2	1.997	3.170	2.033	392.119	766.366
trial-3	2.012	3.170	1.963	397.914	921.355
trial-4	1.997	3.170	1.877	409.038	770.85
trial-5	2.012	3.171	1.897	416.332	775.169
Program
gt_eq_lt_shift	1.279	2.490	1.639	1.284	15.696
trial-2	1.607	2.222	1.508	1.284	13.858
trial-3	1.560	2.223	1.549	1.374	13.846
trial-4	1.513	2.222	1.536	1.279	13.859
trial-5	1.513	2.222	1.544	1.294	13.847
Program
lt_gt_eq_div	1.841	3.107	1.961	22.367	25.975
trial-2	2.044	3.071	1.937	21.414	25.881
trial-3	2.028	3.071	1.952	21.609	25.008
trial-4	2.028	3.069	1.849	21.772	27.017
trial-5	2.059	3.072	1.893	21.86	24.929

Want to test your own mobile device? Visit RunTrial to test out the speed of your device (warning: This might take up to an hour to run).

What patterns can you see in the above table? Do you have more questions after seeing these results? We will spend a good amount of time this course in learning how to properly conduct experiments and benchmark results.

1.1 Definition

An Algorithm is a finite, deterministic and effective problem-solving method suitable for implementation as a computer program.

This course is the study of algorithms, which are the fundamental building blocks of computer science.

1.2 Science

This course, CS 2223 Algorithms, aims to introduce you to the study of the fundamental concepts that relate to computational structures, otherwise known as computer programs. In this course, we are concerned both with ideas and the functional expression of those ideas as computer programs. In this course, we are glad when a computer program produces the correct answer to a problem, but we are excited when we can prove statements about the problem that help us predict the runtime performance of any implementation in any programming language that attempts to solve the same problem.

In presenting the domain of algorithms I would like to remind everyone of similar efforts made by countless scientists over the centuries. For example, what is so special about the following organism?

Figure 2: Drosophila melanogaster

As an aside, if the Ancient Greeks had focused less on geometry and more on computation, who knows how different the world would have been. Squaring the Circle was the ultimate problem: find a square of equal area to a circle using only a finite number of steps with compass and straightedge.

Even Leonardo da Vinci spent countless hours in this pursuit.



Unfortunately, as stated, the problem is impossible.

In this course, we will study a number of fields in computer science that might not seem interesting. Consider the task of sorting in ascending order a collection of arbitrary values. The Sorting domain is one of the most studied fields in all of computer science. Advances in understanding how to write efficient sorting code has led to a deep understanding of Divide and Conquer algorithms.

1.3 Scientific Method

Never forget that this is a computer science class. We approach computational structures with the mind of a scientist. That is:

• Make Observations – Instead of reflecting natural phenomena in the world around us which already exist and are waiting to be observed, we must create computational structures or programs.

• Form Hypotheses – We will focus our attention on the runtime performance of programs on specific classes of input. Note there are many other kinds of questions one could ask (i.e., how hard will it be to maintain this program?).

• Develop Testable Predictions – These hypotheses are used to predict the performance of a program on larger input sizes. The typical scenario is to estimate the cost when doubling the size of the input.

• Gather Data to Test Predictions – You create test cases to validate the correctness of the program. Then you create performance test cases to evaluate runtime performance

• Develop General Theories – By observing numerous computational structures you can develop theories about different approaches. These theories lead to well-designed solution familes, such as Divide and Conquer or Greedy.

1.4 Skills

• Mathematical – Knowledge and ability to use mathematical tools to analyze and prove statements about algorithms.

• Data Structures – Knowledge and ability to work with the fundamental data structures in computer science. This work is performed on mutable data structures, which differs from what you may have done in CS 2102.

• Algorithm Families – Knowledge of key algorithmic families. Ability to analyze performance and correctess of algorithms. Ability to implement and benchmark performance of algorithms.

1.5 Daily Exercises

Each day I will present you with the opportunity to exercise and further develop your problem solving skills. These daily exercises are intended to give you the chance to spend 20 minutes or so thinking about a problem and trying to solve it. I truly believe that you can improve your problem solving abilities with daily practice. I can’t grade these daily questions, though I will post my own solutions at the start of the next class so you can review your answer.

Sometimes I will assign a daily exercise that is "open-ended" without any expectation of how long it will take you to solve the problem. These will be used sparingly, but will be used over a sequence of days to uncover more details about a problem which will help sharpen your attempts at solving the problem.

1.5.1 Closed form formulas

In this course you will become familiar with mathematical tools used to analyze the performance of algorithms. Try your skills at the following question. Later in the course I will conduct similar analyses for algorithmic questions:

Can you come up with a formula that represents the sum of the first n cubic numbers? That is, what is Sn = Σ i3 for i=1 to n?

To get you started, the first few terms of Sn are 1, 9, 36, 100, ...

You should be able to come up with a formula for Sn that uses just n. Take notes in your course note book as you attempt to solve this problem. As an added challenge, can you come up with another way to rephrase problem with a Σ, i and n?

1.5.2 Anagrams

You may find yourself stuck trying to work out some tricky bit of logic. Instead of trying to solve the problem "all at once" you need to make smaller progress or try to find an instance of a solution, before tackling the more general solution. Try your skills at this question:

Rearrange the following letters to form a single English word containing 13 letters.

BLUE MELD SHINE

Take notes in your course notebook as you attempt to solve the problem. Solution will be shown tomorrow.

1.5.3 Thought-provoking Questions

Assume you have a collection of exactly 2n integers from 1 to 2n in order. What if you make the following mistake in the implementation to BINARY ARRAY SEARCH, namely using "<" instead of "<=" as the condition for the while loop:

while (low < high) { ... }

Now search for each of the 2n integers. Can you come up with a closed formula, p(n), that computes the success rate (a number between 0 and 1) of confirming that each integer from 1 to 2n exists in the collection?

It might be easier to empirically determine this result.

As a follow-up, can you compute pe(n), that is, the success rate at finding an even integer from 2 to 2n? Can you compute po(n), that is, the success rate at finding an odd integer from 1 to 2n-1.

1.6 Preparing for Mar 13 2018

Be sure to complete the readings for today as well as Mar 13 2018. As mentioned in class, for each of the following lectures, I have designated a list of 14-18 pages that you must read prior to coming to class. In addition, please review the lecture notes which you can find here 24 hours before the lecture itself.

See if you can answer the following questions. You are given an array of N unique integers.

• What is the fewest number of comparisons to determine the largest integer in an array of n>1 elements. Can you prove this?

• What is the fewest number of comparisons to determine the largest and the 2nd largest integer in an array of n>2 elements. Can you prove this?

1.7 Four Things To Do Today!

• Pick up a course notebook that you can/should use for your class notes as well as ideas towards your homework assignments. We will also use these for in-class activities, so be sure to bring to each class.

• I need a video tape of everyone in the class. I will ask you to face the camera and state your full name (first and last) and then you will spell your last name. This is how I get to know who you are.

• Install Eclipse Neon if you don’t already if you don’t already have one installed. There is a later Oxygen version (released this past year) but for some reason it is incompatible with the fusion.wpi.edu source code system on some platforms. So I recommend you install Neon (or use an older version of Eclipse that you already have installed).

• Make sure you have account on fusion.wpi.edu. Your CCC account credentials will get you in, with no problem. I am not sure how critical this is, but it is easy to do: simply log into fusion.wpi.edu using your CCC credentials.

• Within Eclipse, use Git to retrieve the code that I will be using for the course. I will be populating this repository with additional code and examples. Use repository https://fusion.wpi.edu/anonscm/git/cs2223/cs2223.git. Note that this repository is anonymous to retrieve (since only I will be updating it). I would get used to refreshing this code before each class, so you have the latest code when following the lectures.

  I have posted a video if you have questions

Ok, so that list contains five things. Oh well. See you next class!

1.8 References

[1] Binary_search_algorithm, Wikipedia

1.9 Version : Apr 24 2018

(c) 2018, George Heineman