A scanner, or lexical analyzer, finds the elementary pieces of the program called tokens.
The metalanguage to a scanner generator consists of regular expressions. These regular expressions describe tokens that the generated scanner will then be able to find when a source program is input.
Example 1 shows the regular expressions for the tokens for the components of typical assignment statements.
Example 2 shows an input and output for the generated scanner.
Module 3 discusses the details of generating such a scanner.
