This course studies methods and concepts of computer and communication network modeling and system performance evaluation. Topics include some of stochastic processes, measurement techniques, monitoring tools, statistical analysis of performance experiments, simulation models, analytic modeling and queuing theory, work load characterization and performance evaluation problems. The required background is CS 504, EE 502 or an equivalent background in probability and an interest in learning about computer performance evaluation.
Required Textbooks:
Other Reference Textbooks that look interesting:
Final grades will be computed as follows:
Final grades will reflect the extent to which you have demonstrated understanding of the material, and completed the assigned projects. The base level grade will be a "B" which indicates that the basic objectives on assignments and exams have been met. A grade of "A" will indicate significant achievement beyond the basic objectives and a grade of "C" will indicate not all basic objectives were met, but work was satisfactory for credit. No incomplete grades will be assigned unless there exist exceptional, extenuating circumstances. Similarly, no makeup exams will be given unless there exist exceptional, extenuating circumstances.
There will be two in-class exams. The first is roughly mid-way through the semester and the second is during the last week of the semester. Exams will generally be closed book and closed notes, with the exception of a single sheet of paper (front and back) on which you can write (hand-written, not typed) whatever notes you wish. The majority of each exam will cover basic ideas and objectives of the class with a few questions testing additional understanding and insight.
Late assignments will be be penalized 10% of total assignment value per day (with the weekend counting as one day) or partial day, and no assignments will be accepted after seven days beyond the due date. All assignments are due as indicated when handed out. Exceptions to these rules can only be made beforehand.
Here is the list of objectives you should be able to achieve after this course:
You should check out the assignment timeline to help you plan for doing projects and exams.
Slides from the in-class lectures will be available shortly after they are presented, depending upon how things go. Here is what we have so far:
| Admin | ppt | |
| Introduction (Ch 1-2) | ppt | |
| Statistics (Ch 12-15) | ppt | |
| Selection of Techniques and Metrics (Ch 3) | ppt | |
| Types of Workloads (Ch 4) | ppt | |
| Workload Characterization Techniques (Ch 6) | ppt | |
| Data Presentation (Ch 10-11) | ppt | |
| Monitors (Ch 7) | ppt | |
| Capacity Planning and Benchmarking (Ch 9) | ppt | |
| Experimental Design (Ch 16-17) | ppt | |
| Simulation (Ch 24-25) | ppt | |
| Queuing Theory (Ch 30-31) | ppt | |
| Experiments | ppt | |
| Group Work | - | ppt |
[GS95] John Gustafson and Quinn Snell. HINT: A New Way to Measure Computer Performance, In Proceedings of the Fifth Annual Hawaii International Conference on System Sciences (HICSS), Honolulu, Hawaii, USA, 1995. (slides: ppt, pdf)
[Hen00] John L. Henning. SPEC CPU2000: Measuring CPU Performance in the Net Millennium, IEEE Computer, July 2000. (slides: ppt, pdf)
[LHPT95] P. Lukowicz, E. Heinz, L. Prechelt, W. Tichy. "Experimental Evaluation in Computer Science: A Quantitative Study", Journal of Systems and Software, 28(1):9-18, January 1995. (slides: ppt, pdf)
[WS00] A. Watson and M. A. Sasse. "The Good, the Bad, and the Muffled: the Impact of Different Degradations on Internet Speech", In Proceedings of ACM Multimedia, Los Angeles, CA, USA, November 2000. (slides: ppt, pdf)
There will be 4 projects related to performance evaluation. You must work individually for the projects unless otherwise noted. Projects will be posted here as they are developed. Here is what we have so far:
Project 1: The Dog and the Mailman, (Feb 6) (slides)
Project 2: Network Performance versus TCP Receiver Window Size, (Feb 27) (slides)
Project 3: Benchmarking Benchmarks, (Apr 02) (slides)
In this section are any code samples discussed in class, practice exams or any other demonstration-type class materials.
Final exam stuff:
Mid-term exam stuff:
There are two professional societies dedicated to performance evaluation. They are:
There are numerous benchmarks available for computer system performance evaluation. A few are:
Traces of actual computer and network behavior can be invaluable for driving simulations and building analytic models. Here are a few:
![[Return to the WPI Homepage]](http://web.cs.wpi.edu/images/seal.gif){kind=small}
![[Return to the CS Homepage]](http://web.cs.wpi.edu/images/new_cs.gif){kind=small}
![[Return to Mark Claypool's Homepage]](http://web.cs.wpi.edu/~claypool/gif/mark_thumb.gif){kind=small}