We will use Weka's Naive Bayes and Bayesian Net classifiers to contruct models for this dataset. Assume that the classification target is "veridex_risk". During model construction, use the %split test option, with 90% split. That is, 90% of randomly selected data instances from the dataset are used to construct the model and the remaining 10% of the data instances are used to test the model.

1. (10 points) Construct Naive Bayes models of the dataset using 90% split. Click on "More options ...", to select "Output predictions" (choose say plain text), and to choose a value for the Random seed (initially use value = 1). Repeat the experiment 3 times with seeds 1, 23, 62. For each of the 3 experiments, record in your report the conditional probability values output by Weka under "Naive Bayes Classifier", the accuracy and the confusion matrix obtained with the model over the 10% test set. Report any interesting observations about the results of each experiment and across the 3 experiments.

2. (15 points) Construct Bayesian Nets over this dataset using 90% split. Use K2 as the algorithm to construct the topology of the Bayesian net. Vary the values of "initAsNaiveBayes", "maxNrOfParents", and "randomOrder" parameters of the K2 algorithm. For each experiment, include in your report the obtained Bayesian net (right-click on the experiment on the left hand-side window, and select "Visualize graph"), the accuracy, the confusion matrix, and any interesting observations on the conditional probability tables (click on the nodes in the graph visualization). Report any interesting observations about the results of each experiment and across the 3 experiments.

3. (5 points) Compare the results obtained with Bayesian Nets and with Naive Bayes.

1. (50 points) Using the fair/loaded coin example discussed in class (see slide 14 of Ydo Wexler & Dan Geiger's Markov Chain Tutorial), where there are 2 hidden states C (fair coin) and D (loaded coin), each one producing H (heads) or T (tails), together with the following probabilities:

   Transition probabilities:
       C     D
   C  0.9   0.1
   D  0.1   0.9
   
   Emission probabilities:
   
       H     T
   C  0.5   0.5
   D  0.75  0.25
   

   Assume that both hidden states are equally likely to be the initial state. Represent this by including a "fake" Start state that has no emissions, and has one transition to C and one transition to D, each one with 0.5 probability. [For this problem, it would be very useful for you to explore all the resources on Hidden Markov Models posted on the course webpage.]

   Follow the Forward and the Backward algorithms by hand for the following observed sequence x = TTHH. Show your work and record intermediate results of the dynamic programming algorithms in tables F and B, as the algorithms would. Note that:

   • the F table would have an additional column 0, and additional row 0, corresponding to the fake Start state.
   • as discussed in class, multiplying small probabilities can create underflow errors. If you do run into underflow errors, redo your calculations in log₂ space (see Prof. Kellis' Algorithms for Computational Biology course (MIT) lecture notes and Prof. Mneimneh's Computational Biology course (Hunter College) lecture notes), or scaling (see Rabiner's tutorial Section V).

   Once that those tables have been completed, calculate the probability that the 3rd hidden state visited (i.e., the state that produced the leftmost H) was C (the fair coin). That is, calculate:

   p(s₃= C | TTHH) = ?

   Remember that p(s₃= C | TTHH) = p(TTHH, s₃=C)/p(TTHH). Don't forget to divide by p(TTHH), whose value you can easily calculate from the Forward table.

2. (100 points + 100 extra credit points) This part of the assignment is based on a homework assignment from Prof. Subramanian's "From Sequence to Structure: An Introduction to Computational Biology" course (Rice Univ.). Take a look at Prof. Subramanian's useful Markov models and HMMs Matlab demos.

   Follow the instructions in the homework assignment, starting on page 2 (you do NOT need to work on parts (a)-(d) on page 1). Include in your written report answers to parts (a)-(g) on pages 3-4. Credit points are as follows: (a)-(b) 10 points each; (c)-(f) 15 points each; (g)20 points. Please note that the data files have been changed since the date of the above assignment (2009). Current information (as of Sept. 2011) is included below.

   You can download all the needed data files from http://www.ncbi.nlm.nih.gov/mapview/maps.cgi?taxid=9606&chr=22. For simplicity, I include the files below (current as of Sept. 2011):

   • ncbimapBuild_37.2_22_cpg_0K.txt: CpG island locations on the human chromosome 22.
     (Taken from the NCBI's CpG island for human chromosome 22 webpage. Also accesible from "CpG island" on the NCBI's human chromosome 22 webpage.)

   • Human chomosome 22's contigs 1, 2, and 3:
     (You can find links to the original files by looking for "Contigs" on the NCBI's human chromosome 22 webpage.)

     Region Displayed: 0-51M bp Download/View Sequence/Evidence Download Data
     Total Contigs On Chromosome: 4
     Contigs in Region: 0
     	start 		stop		Symbol		O
     	16050001 	16697850	NT_028395.3	+
     	16847851 	20509431	NT_011519.10	+
     	20609432 	50364777	NT_011520.12	+
     	50414778 	51244566	NT_011526.7	+
     

     1. genbank_human_chr22_contig1.fasta
     2. genbank_human_chr22_contig2.fasta
     3. genbank_human_chr22_contig3.fasta

   Extra Credit (100 points): Repeat the same steps above, but using a HMM with 8 hidden states: A+, C+, G+, T+, A-, C-, G-, T- where the "+" states represent the nucleotides in a CpG island, and the "-" states represent the nucleotides in regular DNA. Each state emits only the corresponding nucleotide. That is, A+ and A- emit A; C+ and C- emit C; etc. Include transitions from each of the states to all the other 7 states.