• The dataset for this project is the same we used for Problem Set 1: GSE7390_transbig2006affy_demo.txt (this is a local copy of the dataset).

  This dataset is part of the NCBI's GSE7390 Data Set. This dataset contains information for 198 untreated patients of the TRANSBIG validation study. See the README.txt file that describes the dataset. (To simplify the data dowloading process, you can find the same files described above in a local copy of the dataset.)

  Remove the following dataset attributes from consideration: "samplename", "id", "geo_accn", "filename", and "hospital".

Apply the following clustering methods to your dataset. Describe in your written report the results of your experiments.

1. (50 points) K-means.
   In this part, you will experiment both with Weka and with Matlab (see the Statistics Toolbox), applying K-means to the dataset above.

   1. (10 points) Run a total of at least 5 different experiments in Weka and at least 5 different experiments in Matlab varying the following parameters:
      • different values of k: use at least 2 different values of k bewtween 2 and 6,
      • different distance metrics: Euclidean and Manhattan (= cityblock).
      • different seed values (in Weka): 10, 27; and different 'start' options in Matlab: 'sample', 'uniform', 'cluster'.
      • with and without normalizing each of the attributes before clustering.
      • in Weka, use both .arff versions of the dataset (one with numeric and one with nominal values for the attributes "Histtype", "Angioinv", "Lymp_infil", and "grade").
      Note: You don't have to run all possible experiments varying all parameters above one by one. But try to run a good set of experiments so that you can see the effect of these parameters on the resulting clustering.

   2. Include in your written report (summarize parts 1 and 2 below in a table):
      1. (10 points) a summary of the each experiment and its results: parameters used, number of instances in each cluster, centroids, and error value ("Within cluster sum of squared errors" in Weka and the sum of values in the "sumd" vector in Matlab) for each clustering.
      2. (5 points) any interesting findings or observations you have about each clustering.
      3. Select the best clustering (i.e., lowest error value) obtained with each tool (Weka and Matlab). Let's call the best Weka clustering you obtained WCbest, and the best Matlab clustering you obtained MCbest.
         1. (5 points) Produce Scatterplot and MultiDimensional Scaling (MDS) visualizations of WCbest and MCbest.
         2. Investigate what "Cluster Purity" and "Normalized Mutual Information (NMI)" are. See for example Evaluation of clustering which is part of the online book "Introduction to Information Retrieval" By Christopher D. Manning, Prabhakar Raghavan & Hinrich Schütze.
         3. (5 points) Include a brief description of these metrics in your report.
         4. (10 points) Calculate Purity and NMI for each clustering WCbest and MCbest with respect to the class/target attribute "veridex_risk".
         5. (5 points) Based on the above comparisons, what conclusions can you draw about K-means clustering of this dataset? Elaborate on your answer.

2. (40 points) Hierachical Clustering.
   In this part, you will experiment both with Weka and with Matlab (see the Statistics Toolbox), applying K-means to the dataset above.

   1. (10 points) Run a total of at least 5 different experiments in Weka and at least 5 different experiments in Matlab varying the following parameters:
      • different distance metrics: Euclidean and Manhattan (= cityblock).
      • different link types: use at least 3 different link types: single, complete, average, centroid, Ward.
      • with and without normalizing each of the attributes before clustering.
      • in Weka, use both .arff versions of the dataset (one with numeric and one with nominal values for the attributes "Histtype", "Angioinv", "Lymp_infil", and "grade").
      Note: You don't have to run all possible experiments varying all parameters above one by one. But try to run a good set of experiments so that you can see the effect of these parameters on the resulting clustering.

   2. Include in your written report (summarize parts 1 and 2 below in a table):
      1. (10 points) a summary of each experiment and its results.
      2. (10 points) any interesting observations you can make about the results of the experiments.
      3. (10 points) Pick your favorite hierachical clustering experiment. Let WC and MC be the hierachical clusterings obtained from Weka and Matlab respectively for the options/settings of your favority experiment. Include in your report a tree visualization (and other visualizations if you wish) of WC and MC.
      4. (5 points) Based on the above results and visualizations, what conclusions can you draw about hierachical clustering of this dataset? Elaborate on your answer.

3. (20 points) EM Clustering.

   1. Consider the 1-dimensional dataset dataset_for_EM.txt that I generated at random using 2 normal distributions, as discussed in class. I won't tell you what means and standard deviations I used, but you'll try to estimate these parameters using the EM algorithm.
      1. (5 points) Upload this dataset onto Matlab and use Gaussian Mixture Models Clustering based on Gaussian mixture models using the EM algorithm to cluster this dataset into 2 clusters. What mean and stardard deviation did you obtain for each of the 2 clusters?
      2. (5 points) Upload this dataset onto Weka and use the EM algorithm to cluster this dataset into 2 clusters. What mean and stardard deviation did you obtain for each of the 2 clusters?
      3. (5 points) Compare the above results from Matlab and Weka. In addition, use visualizations to see the clusters.

   2. (10 points)See my examples on how to generate random data using normal distributions in Matlab. Mimic this example to generate a dataset with 3 different normal distributions. Upload the resulting dataset onto Matlab and/or Weka, and apply EM clustering to see how close Matlab and/or Weka can determine the correct parameter distributions (means and standard deviations). Repeat the experiment a couple of times varying the means and standard deviations of the 3 input sets in the dataset. Include in your report a summary of the experiments you ran, the results, and any useful visualizations. Elaborate on how the proximity between the means and the values of the standard deviations (which influence the degree of overlap among the clusters) affect the accuracy of the clustering results produced by Weka and/or Matlab.