Performance/Benchmarking

Here is an outline of topics to discuss:

• Benchmarking Overview, IEEE Computer Dec 1990.

  SPEC Benchmarks up to CPU2000. CPU95 benchmark retired in June 2000. Article in IEEE Computer July 2000 gives results from ``benchathon'' held in February 1999. Most recent benchmarks on High Performance Computing (HPC2002) and Java Application Server (jAppServer2002).

• ``The Measure Performance of Personal Computer Operating Systems,'' ACM Transactions on Computer Systems, Feb 1996.
  Slides: http://www.cs.wpi.edu/~cs535/s03/chen:tocs96

• ``Using Latency to Evaluate Interactive System Performance,'' OSDI Conference, 1996
  Slides: http://www.cs.wpi.edu/~cs535/s03/endo:osdi96

• Web benchmarks:
  SPECmarks: http://www.specbench.org/

  WebStone: http://www.mindcraft.com/webstone/

• ``Brittle Benchmarks in Operating Systems Research''
  http://www.cs.wpi.edu/~cs535/s03/mogul:hotos99/

• ``Supporting Time-Sensitive Applications on a Commodity OS''
  http://www.cs.wpi.edu/~cs535/s03/goel:osdi02/