CS 2223 Mar 13 2023

Lecture Path: 01
Next

For the remainder of this course, I expect you to have completed all readings and demonstrations prior to the start of the lecture on which they are listed.

Before presenting this lecture, I will review the course structure.

Diamonds are found only in the dark places of the earth, truths are found only in the depths of thought.
Victor Hugo
Les Miserables

1 A Journey Of A Thousand Miles Begins With A Single Step

A perfect algorithm is like an artistic study in miniature. Consider the amazing portrait of Ginevra de’ Benci by Leonardo da Vinci [1474/1478]

Leonardo da Vinci painted this portrait – only 15x15 inches in size – when he was just 21 years old. It signaled a revolution in portraiture, fundamentally changing the way artists approached the subject. There is nothing extra in this artwork. Every brushstroke is meaningful and I encourage you to "zoom in" on the portrait. Click on the link below the image and you will be brought to the National Gallery of Art web site where you can click on the image there to explore. Right-click to see the mouse commands; you can double-click to zoom in and then move around to see the incredible detail of the picture.

Now let’s look at another elegant miniature, this time the BINARY ARRAY SEARCH algorithm which determine whether a sorted array contains a target item. Here is a description from Jon Bentley’s amazing book, Programming Pearls:

BINARY [ARRAY] SEARCH solves the problem [of searching within a pre-sorted array] by keeping track of a range within the array in which T [i.e., the sought value] must be if it is anywhere in the array. Initially, the range is the entire array. The range is shrunk by comparing its middle element to T and discarding half the range. The process continues until T is discovered in the array, or until the range in which it must lie is known to be empty. In an N-element table, the search uses roughly log2(N) comparisons.

This algorithm, so briefly stated, is the foundation of so many efficient algorithms. Easily stated, but not so easily implemented. Bentley reports that given this description, 90% of professional programmers are unable to code this algorithm given several hours. What follows is a Java implementation:

Given the above program, I have some simple questions to ask:

Important questions are hilighted in orange. Pay attention to these!

• Is the while loop guaranteed to terminate?

• How many times will the while loop execute?

• Can you prove this code is correct?

By the end of this course, I will teach you how to look at code fragments to understand the elegance and beauty of algorithms. The simplicity of BINARY ARRAY SEARCH is legendary. And just as important, one needs correct implementations that are validated to handle all cases, especially the boundary cases that make it hard to translate algorithms into code.

There are some subtle features of the above code that you might not notice at first glance:

• Look at the order of the if statements. In the code above it first checks less than (lt), then greater than (gt) before concluding the values are equal (eq). Does it matter what order these checks are made? It sure does, as you can see below.

• Does this midpoint computation have any unexpected pitfalls?

  Look at the computation for the midpoint mid. This uses a standard integer truncation division operator. If you replace with a right-shift bitwise operator >> 1 the performance increases noticeably

The following table reports the result of an experiment. We generate a sorted array of 65,536 unique integers drawn from the range [-16777216, 16777216]. We then create a new list of 65,536 target integers in the range [-33554432, 33554432]. The target range is wider since we want to ensure we can search for numbers that are both too low and too high.

Any thoughts on the probability that a target search actually finds a value from the original sorted array? That is, of the T=33,554,432 individual searches, how many times will a random target value be successfully found in the original sorted array? Can you come up with a formula that estimates this number based on the definition of T, the initial sorted array, and the random target values?

We then want to generate a single trial run. Because computers are so fast these days, in each run we carry out the following task 512 times: search for each of the 65,536 target values in order. Thus there will be a total of 512 x 65,536 = 33,554,432 invocations of the contains method. All reported results are in seconds.

1.1 Definition

An Algorithm is a finite, deterministic and effective problem-solving method suitable for implementation as a computer program.

This course is the study of algorithms, which are the fundamental building blocks of computer science.

1.2 Science

This course, CS 2223 Algorithms, aims to introduce you to the study of the fundamental concepts that relate to computational structures, otherwise known as computer programs. In this course, we are concerned both with ideas and the functional expression of those ideas as computer programs. In this course, we are glad when a computer program produces the correct answer to a problem, but we are excited when we can prove statements about the problem that help us predict the runtime performance of any implementation in any programming language that attempts to solve the same problem.

In presenting the domain of algorithms I would like to remind everyone of similar efforts made by countless scientists over the centuries. For example, what is so special about the following organism?

If you have ever taken a course in biology then you should be familiar with the fruit fly, one of the most studied organisms in scientific history. Researchers study fruit flies because they contain "a wealth of biological data that makes them attractive to study as examples for other species and other natural phenomena that are more difficult to study directly." Drosophila has 13,600 genes and it has been fully sequenced. Scientists study Drosophila not because it is important in and of itself, but because it provides a platform for study.

The power of modeling is most evident when considering the following scrap of paper. Can anyone recognize what it is?

Figure 3: Back of envelope computation

As an aside, if the Ancient Greeks had focused less on geometry and more on computation, who knows how different the world would have been. Squaring the Circle was the ultimate problem: find a square of equal area to a circle using only a finite number of steps with compass and straightedge.

Even Leonardo da Vinci spent countless hours in this pursuit.



Unfortunately, as stated, the problem is impossible.

In this course, we will study a number of fields in computer science that might not seem interesting. Consider the task of sorting in ascending order a collection of arbitrary values. The Sorting domain is one of the most studied fields in all of computer science. Advances in understanding how to write efficient sorting code has led to a deep understanding of Divide and Conquer algorithms. Studying sorting has led to sophisticated mathematical analysis that we will cover (in part) in this course.

1.3 Scientific Method

Never forget that this is a computer science class. We approach computational structures with the mind of a scientist. That is:

• Make Observations – Instead of reflecting natural phenomena in the world around us which already exist and are waiting to be observed, we must create computational structures or programs.

• Form Hypotheses – We will focus our attention on the runtime performance of programs on specific classes of input. Note there are many other kinds of questions one could ask (i.e., how hard will it be to maintain this program?).

• Develop Testable Predictions – These hypotheses are used to predict the performance of a program on larger input sizes. The typical scenario is to estimate the cost when doubling the size of the input.

• Gather Data to Test Predictions – You create test cases to validate the correctness of the program. Then you create performance test cases to evaluate runtime performance

• Develop General Theories – By observing numerous computational structures you can develop theories about different approaches. These theories lead to well-designed solution familes, such as Divide and Conquer, Greedy, or Dynamic Programming.

1.4 Skills

• Mathematical – Knowledge and ability to use mathematical tools to analyze and prove statements about algorithms.

• Data Structures – Knowledge and ability to work with the fundamental data structures in computer science. This work is performed on mutable data structures, which differs from what you may have done in CS 2102.

• Algorithm Families – Knowledge of key algorithmic families. Ability to analyze performance and correctess of algorithms. Ability to implement and benchmark performance of algorithms.

1.5 Lecture Challenges

Each day I will present you with the opportunity to exercise and further develop your problem solving skills. These lecture challenges are intended to give you the chance to spend 20 minutes or so thinking about a problem and trying to solve it. I truly believe that you can improve your problem solving abilities with daily practice. I can’t grade these challenges, though I will post my own solutions at the start of the next class so you can review your answer.

Sometimes I will assign a challenge that is "open-ended" without any expectation of how long it will take you to solve the problem. These will be used sparingly, but will be used over a sequence of days to uncover more details about a problem which will help sharpen your attempts at solving the problem.

1.5.1 Closed form formulas

In this course you will become familiar with mathematical tools used to analyze the performance of algorithms. Try your skills at the following question. Later in the course I will conduct similar analyses for algorithmic questions:

Can you come up with a formula that represents the sum of the first n cubic numbers? That is, what is Sn = Σ i3 for i=1 to n?

To get you started, the first few terms of Sn are 1, 9, 36, 100, ...

You should be able to come up with a formula for Sn that uses just n. Take notes in your course note book as you attempt to solve this problem. As an added challenge, can you come up with another way to rephrase problem with a Σ, i and n?

1.5.2 Binary Search with duplicates

Please spend some time thinking about this problem. I will bring this topic up in lecture TOMORROW.

Computer Science discovers new knowledge by making small changes to a problem. What if you wanted to know how many copies of a target value exist in a sorted array? For example, given the sorted array [1, 3, 5, 5, 5, 5, 9], the value 5 is repeated 4 times. Naturally, if the target value doesn’t exist in the array, you should return 0.

Can you think of a way to adapt the existing logic for BINARY ARRAY SEARCH to solve this problem efficiently? I will start tomorrow with this adaptation, but I want to give you a night to think about it yourself.

1.5.3 Thought-provoking Questions

Assume you have an array of exactly 2n integers from 1 to 2n in order. What if you make the following mistake in the implementation to BINARY ARRAY SEARCH, namely using "<" instead of "<=" as the condition for the while loop:

while (lo < hi) { ... }

Now search for each of the 2n integers. Can you come up with a closed formula, p(n), that computes the success rate (a number between 0 and 1) of confirming that a randomly selected integer from 1 to 2n is found by this defective code?

1.6 Preparing for Mar 14 2023

Be sure to complete the readings for today as well as Mar 14 2023. As mentioned in class, for each of the following lectures, I have designated a list of 14-18 pages that you must read prior to coming to class. In addition, please review the lecture notes which you can find here 24 hours before the lecture itself.

I will address tomorrow these two questions. What do you think the answers are?

• What is the fewest number of comparisons to determine the largest integer in an unordered array containing N values? Can you prove this?

• What is the fewest number of comparisons to determine the largest and the 2nd largest integer in an unordered array containing N values? Can you prove this?

1.7 Three Things To Do Today!

• Install Eclipse 2022-12 or 2021-03 if you don’t already have one installed. I recommend you install the latest version of Eclipse (or use a slightly older version of Eclipse that you already have installed). Be sure you have latest Java version 1.8 installed on your computer (even though there are later versions, I will be sure to stick to 1.8).

• Within Eclipse, use Git to retrieve the code that I will be using for the course. I will be populating this repository with additional code and examples. The code base is hosted by a newly launched CS 2223 GitLab Repository. NOTE: this can only be accessed from within the WPI network, so you need to either use VPN, remote desktop, or bring your laptop to campus and retrieve files from there.

  You may have to install Global Protect to be able to access the repository remotely. I might have to do ths same thing! And if so, I’ll post a video.

  This repository can be retrieved by anyone with a WPI account who has logged into the gitlab03.wpi.edu site using single-sign on credentials (since only I will be updating it). In the current process, you request to log in, only be told that your account is blocked, and then it is manually unblocked. I hope they can streamline this process but for now, this is what happens.

  I would get used to refreshing this code before each class, so you have the latest code when following the lectures.

  I will post a video to get you started.

• Decide how you will access the readings I’ve assigned from the book. I know text books are expensive; I’ve used this book for several years and I think the author has done a great job in clearly presenting the material. Perhaps find a used copy? or share a copy with a classmate? Each day there will be assigned reading which I will cover in my own lecture notes, but it is simply more comprehensive than I have time to present in my limited class time.

On to the next class!

1.8 References

[1] Binary_search_algorithm, Wikipedia

1.9 Version : 2023/03/10

(c) 2023, George T. Heineman