• Assignment: Complete Part I of the assignment described in the data and code webpage (under "Documentation") by Tom Mitchell. Include answers to the all the questions in that assignment in your written report. For your convenience, here is a PDF version of the code documentation and of Mitchell's assignment.

• Code: You can use
  • the C code for the neural network error backpropagation algorithm from Chapter 4 of the Mitchell's textbook, or
  • your own code.
  Your code must run on the CS or CCC Unix machines. Make sure that the code you use correctly computes the error function of the neural net when more than one output node is used. For that you need to read the code and understand it in detail.

• Training/Test Instances: Use the "quarter"-size collection of pictures provided at the CMU site. You can find copies of those pictures in our course public directory /cs/cs4341/Proj4/faces_4/ on the CCC machines.

  Also, thanks to Nathan Sheldon, here are the lists with the appropriate (not CMU) directory paths. I have also left a copy of that file in /cs/cs4341/Proj4/proj4_lists.txt

Consider the following approach to training a neural net that uses genetic algorithms instead of the error backpropagation algorithm. For simplicity, assume that we are training a 2-layer, feedforward neural net with 4 inputs, 1 hidden layer with 2 hidden nodes, and one output node. We have a collection of say n training examples to train the net.

• Each individual in the population encodes a value for the weight of each and everyone of the connections in the neural net.
• The fitness of an individual must represent how close the output computed by the net is to the desired output over all the training examples. The higher the fitness of an individual, the better the corresponding neural net computes the target function.
• Cross-over is done at the weight level. That is, cross-over points will be restricted to occur only within (the codification of) two weights but not inside (the codification of) one weight.

Solve the following problems:

1. Describe precisely the bit-string encoding of individuals.
2. Suppose that the initial population consists of 100 individuals. Describe in detail how you would obtain those initial individuals.
3. Describe precisely the fitness function.
4. Describe precisely the selection, mutation, and cross-over operators.
5. Describe precisely the genetic algorithm (or sequence of steps) that you would follow to train the neural net using this evolutionary approach. What are the termination conditions of the algorithm?
6. Describe precisely what the output of your algorithm is.

Consider the following dataset that specifies the type of contact lenses that is prescribed to a patient based on the patient's age, spectacle prescription, astigmatism, and tear production rate. Use information theory, more precisely entropy, to construct a minimal decision tree that predicts the type of contact lenses that will be prescribed to a patient based on the patient's attributes.
SHOW EACH STEP OF THE CALCULATIONS and the resulting tree.

  Attribute age                  values: {young, pre-presbyopic, presbyopic}
  Attribute spectacle-prescrip   values: {myope, hypermetrope}
  Attribute astigmatism          values: {no, yes}
  Attribute tear-prod-rate       values: {reduced, normal}
  Attribute contact-lenses       values: {soft, hard, none}
  
  
  age              spectacle-     astigmatism tear-prod-rate contact-lenses
                   prescription
  
  young            myope          no          reduced         none
  young            myope          no          normal          soft
  young            myope          yes         reduced         none
  young            myope          yes         normal          hard
  young            hypermetrope   no          reduced         none
  young            hypermetrope   no          normal          soft
  young            hypermetrope   yes         reduced         none
  young            hypermetrope   yes         normal          hard
  pre-presbyopic   myope          no          reduced         none
  pre-presbyopic   myope          no          normal          soft
  pre-presbyopic   myope          yes         reduced         none
  pre-presbyopic   myope          yes         normal          hard
  pre-presbyopic   hypermetrope   no          reduced         none
  pre-presbyopic   hypermetrope   no          normal          soft
  pre-presbyopic   hypermetrope   yes         reduced         none
  pre-presbyopic   hypermetrope   yes         normal          none
  presbyopic       myope          no          reduced         none
  presbyopic       myope          no          normal          none
  presbyopic       myope          yes         reduced         none
  presbyopic       myope          yes         normal          hard
  presbyopic       hypermetrope   no          reduced         none
  presbyopic       hypermetrope   no          normal          soft
  presbyopic       hypermetrope   yes         reduced         none
  presbyopic       hypermetrope   yes         normal          none
  

• If you develop your own error backpropagation code, you should submit the source code of your program documented using the Departmental Documentation Standard.

• A file proj4_ann.txt with your written report for this part of the assignment. Your report should discuss the following issues:
  1. answers to all questions in Part I of Mitchell's assignment (for your convenience, here is a PDF version of the code documentation and of Mitchell's assignment),
  2. adaptation of the code (if any) or a description of your own code,
  3. the experiments you ran with the system,
  4. the topology (number of units in each hidden layer), number of iterations of the error backpropagation algorithm, and all other input parameters that you used for each of your experiments as well as the output printed out by the error backpropagation code, including the accuracy of each of the neural nets,
  5. strengths and the weaknesses of the system.
  Your report should also include a short user manual explaining how to install, run, and use your system (if different from the CMU package).