12.2.9 Benchmarks

Benchmark programs measure the efficiency of compiler (and machines). Some well-known benchmark programs are Whetstone (Curnow and Wichmann, 1976; Weicker, 1990) and Dhrystone (Weicker, 1984, 1990). Whetstone was developed for scientific computing applications and is heavy in floating-point operations. Dhrystone was developed for non-numerical applications and is heavy in string operations.

Another frequently used measurment is MIPS (Millions of Instructions Per Second). Without a base value to compare to, however, this becomes "Meaningless Indication of Processor Speed". For example, a one statement instruction on a non-RISC processor might require four statements on a RISC machine; it would be incorrect, if the execution speed were the same, to say that the RISC machine is four times as fast.

In addition, language-specific benchmarks are common. This is especially true of languages like Ada (Clapp et al., 1988). Measurement of the performance of specific language features is useful in specific applications.

More useful benchmark programs are those which are actually used, especially if the environment in which the compiler is to be run is known. For example, YACC is a compute-intensive program and is often run as a benchmark program. Other familiar software such as L_AT_EX, troff, awk and various compilers are often used. Special programs such as the Sieve of Eratosthenes and Quicksort are sometimes used since they can be recorded into whatever language is being compiled.