See solutions by Artem Gritsenko.

1. Constructing Naive Bayes and Bayesian Nets
   Consider the following dataset.

   @relation weather-weka.filters.unsupervised.attribute.Remove-R2
   
   @attribute outlook {sunny,overcast,rainy}
   @attribute humidity {low,normal,high}
   @attribute windy {TRUE,FALSE}
   @attribute play {yes,no}
   
   @data
   ( 1)  sunny,     normal,  FALSE,  no
   ( 2)  sunny,     high,    TRUE,   no
   ( 3)  overcast,  normal,  FALSE,  yes
   ( 4)  rainy,     high,    FALSE,  yes
   ( 5)  rainy,     normal,  FALSE,  yes
   ( 6)  rainy,     low,     TRUE,   no
   ( 7)  overcast,  low,     TRUE,   yes
   ( 8)  sunny,     high,    FALSE,  no
   ( 9)  sunny,     low,     FALSE,  yes
   (10)  rainy,     normal,  FALSE,  yes
   (11)  sunny,     low,     TRUE,   yes
   (12)  overcast,  high,    TRUE,   yes
   (13)  overcast,  normal,  FALSE,  yes
   (14)  rainy,     high,    TRUE,   no
   

   where the play attribute is the classification target.

   1. (15 points) Construct the full naive Bayes model for this dataset. Show all the steps of the calculations. Draw the resulting graph and the Conditional Probability Table (CPT) associated with each node in the graph.

   2. (5 points) Use your model above to classify the following data instance (note that this data instance contains a missing value). Explain your work.

      outlook=sunny, humidity=normal, windy=?
      

   3. (5 points) Consider the following Bayesian net over the dataset above. Construct the conditional probability table for the outlook node. Show your work.

      

2. An existing real-world application of Naive Bayes or Bayesian Nets
   Each group should search for an existing real-world successful application of Naive Bayes models or Bayesian Nets. This sucessful story should be about mining these Bayesian models from data to discover novel and useful patterns that have made a difference in a certain industry or field. The application domain is up to the group (e.g., finance, sports, healthcare, science, ...). The application must be a current application (ideally not older than 5 years).

   Your homework report for this part should contain (at most 1 page):

   • (10 points) Name and description of the chosen application.
   • (10 points) Detailed description of how this application uses Naive Bayes or Bayesian Networks.
   • (5 points) List of (recent) references and sources that you used to identify and investigate this application.

• Project Instructions: THOROUGHLY READ AND FOLLOW THE PROJECT GUIDELINES. These guidelines contain detailed information about how to structure your project, and how to prepare your written and oral reports. Note that the project guidelines specify strict page limits for your written report.

• Data Mining Technique(s): We will run experiment using the following techniques:
  • Pre-processing Techniques:
    • Feature selection, feature creation, dimensionality reduction, noise reduction, attribute discretization, ...

  • Data Mining Techniques (include descriptions of the Weka code implementing these techniques in your written report):
    • Naive Bayes
    • Bayesian Nets
    Given that these techniques are able to handle numeric attributes and missing values directly, make sure to run some experiments with no pre-processing and some experiments with pre-processing, and compare your results. Experiment also with different parameter values to see how they affect the graphs and CPTs of the models, and their classification performance.

• Advanced Techniques:
  • You can consider using advanced techniques to improve the accuracy of your predictions. For instace, you can try ensemble methods (see Section 5.6 of your textbook), ways to deal with inbalanced classification targets (see Section 5.7 of your textbook), etc.

In particular,

• Use the data in the file "adult.data", and the description of the data in the file "adult.names".
• Use the nominal attribute salary (called >50K, .50K in the data files) as the classification target.
• remove the fnlwgt attribute from the dataset.

• Easy Level:

  • Easy Level Part I.
    This is to be a simple guided experimentation, thus little description is needed for preprocessing techniques.

    Create a Bayesian network model using Weka's implementation of NaiveBayes using the default parameters. Use salary as the target attribute.

    Create a Bayesian network model using Weka's implementation of BayesNet using the default parameters. Use salary as the target attribute.

    Examine your two models. Compare and contrast them. Use 10-fold cross-validation to perform an analysis of the classification accuracy. Answer the following questions in your description about this experiment:

    1. Compare the topology of the two models generated. Are they different? If so how? If not, why not?
    2. Examine the conditional probability tables for the attribute race for each of models. How are these tables different? How are they similar?
    3. Compare the accuracy of the two models. Are the accuracies of the models significantly different?

  • Easy Level Part II.
    This is a bit more of a challenge.

    Use modified parameters and preprocessing techniques to generate a NaiveBayes that attempts to classify salary accurately. Provide detailed descriptions about the parameters used to develop your model and/or preprocessing techniques used. One should be able to repeat the experiment from your description.

    Examine the model. Compare and contrast this model against a ZeroR model, a OneR model, models generated in Project 2, and models generated in the challenge above. Answer the following questions in your description about this experiment:

    1. Is this model a better model than the other models? If so, why? If not, why not?
    2. Did you find that using a dataset with no preprocessing but modified parameters produced a better model that using a preprocessed dataset with default parameters? Explain why or why not.
    3. What challenge(s) did you encounter while developing this model? Give a more detailed explanation of how you used preprocessing, postprocessing, or some other technique to overcome a specific challenge.

• Moderate Level: This and the WPI are the big challenges that should spend the most time on.

  Use preprocessing and modified parameters to generate a BayesNet that attempts to accurately classify salary. Provide detailed descriptions about the parameters used to develop your model and/or preprocessing techniques used. One should be able to repeat the experiment from your description. Describe the topology of the model produced.

  Examine the model. Compare and contrast this model against a ZeroR model, a OneR model, NaiveBayes, and models generated in Project 2.

  1. Describe how this model is different from the BayesNet obtained in the "easy" challenge.
  2. What challenge(s) did you encounter while developing this model? Give a more detailed explanation of how you used preprocessing, postprocessing, or some other technique to overcome a specific challenge.
  3. Describe the topology of th net and any anomalies that appeared in the topology. What might these anomalies mean about the data?
  4. Look at the Conditional Probability Tables (CPTs). Discuss any interesting aspects of these CPTs.

• WPI Level: This and the Moderate Level are the big challenges that should spend the most time on.

  Design another experiment with a different goal other than the ones that have appeared previously in this assignment. the more interesting, useful, creative, original, and non-trivial your goal is, the better your grade on this part. Provide detailed descriptions about the parameters used to develop your model and/or preprocessing techniques used. One should be able to repeat the experiment from your description. Describe the topology of the model produced. Compare the performance of your model against ZeroR, OneR, and J4.8 models.

  1. What was your motivation for choosing this goal?
  2. Are there any limitations of the dataset that make this a more challenging experiment? Explain.
  3. Describe the topology of the net and any anomalies that appeared in the topology. What might these anomalies mean about the data?
  4. Look at the Conditional Probability Tables (CPTs). Discuss any interesting aspects of these CPTs.