Due: Friday March 26 Grammar
Due: Friday, April 2 Lexer
Due: Friday, April 16 Parser
Due: Friday, April 30 AST's

Assignment #2

Lexer

10 Points

In this assignment, you will use javacc to generate a lexical analyzer for Tea. The tokens are:

Unlike lex and yacc, the JavaCC tool does not have a separate lexical analyzer generator. We will embed the tokens and the output statements for them within a null parser file.

Here is the outline of the .jj file you will write:



options {
  IGNORE_CASE = false;
  OPTIMIZE_TOKEN_MANAGER = true;
}

PARSER_BEGIN(tea)

import java.io.*;

public class tea {

  public static void main(String[] args) throws FileNotFoundException
  {
    if ( args.length < 1 ) {
       System.out.println("Please pass in the filename for a parameter.");
       System.exit(1);
    }
    
    ASCII_CharStream stream = new ASCII_CharStream(
                                  new FileInputStream(args[0]),0,0);
    Token temp_token = null;
    
    teaTokenManager TkMgr = new teaTokenManager(stream);

    do {
        temp_token = TkMgr.getNextToken();

        switch(temp_token.kind) {
         case IF: System.out.println("IF: " + temp_token.image);           
           break;
         case TYPEDEF: System.out.println("TYPEDEF: " + temp_token.image);
           break;

         .........
        
         default:
           if ( temp_token.kind != EOF )
             System.out.println("OTHER: " + temp_token.image);
           break;
        }
    } while (temp_token.kind != EOF);
  }
}

PARSER_END(tea)

SKIP: /* Whitespace */
{
  "\t"
| "\n"
| "\r"
| " "
}

TOKEN: 
{
  <IF: "if">
| <TYPEDEF: "void" | "int">

.............

| <NUMBER: (["0"-"9"])+>
}

You need to add your tokens within the switch statement to print them out and within the TOKEN declaration.

To run JavaCC:

If you have downloaded it into a /bin directory, the command is:

tea

You then compile the generated tea.java to create a tea.class file which is your lexical analyzer:

javac tea.java To run your generated lexer, type:

input file

Run your generated lexer on:

(a)
void input_a() {
     a = b3;
     xyz = a + b + c - p / q;
     a = xyz * ( p + q );
     p = a - xyz - p;
}

(b)
void input_b() {
     if ( i > j )
       i = i + j;
     else if ( i < j )
       i = 1;
}

(c)
void input_c() {
     while ( i < j && j < k ) {
           k = k + 1;
           while ( i == j )
                 i = i + 2;
     }
}



(d) A program of your choice. It should test the tokens not tested
in the others. 


 Pass in:  the  JavaCC  source file, input and output from the four programs. 


In this, and all assignments, you will be graded on the appearance and 
documentation of your project. Include your token list as part
of the documentation. You should use a laser printer to print. 


 







  





 Send questions and comments to:
Karen Lemone