PROJECT DESCRIPTION

PROJECT ASSIGNMENT

• Code: You can use the clustering methods implemented in the Weka system. You are encouraged to run experiments with the 3 clustering methods discussed in class K-MEANS, COBWEB, and EM (which are available in Weka) and to compare the results from them.

• Data Instances:

  You may restrict your experiments to a subset of the dataset IF Weka cannot handle your whole dataset (this is unlikely though).

  A main part of this project is the PREPROCESSING of your dataset. You must apply relevant concept hierarchies and generalizations (e.g. using the results of previous mining tasks) to your dataset before clustering it. Your report should contained a detailed description of the preprocessing of your dataset and justifications of the steps you followed. If Weka does not provide the functionality you need to preprocess your data as you need to obtain useful patterns, preprocess the data yourself either by writing the necessary filters (you can incorporate them in Weka if you wish).

REPORT AND DUE DATE

• Written Report. Your written report is due at 3:00 pm. Please hand it in at the beginning of class. Your report should contain the following sections with the corresponding discussions:

  1. Code Description: Explain in some detail the algorithm underlying each of the clustering methods used. Describe each method in terms of the input it receives and the output it produces, and the steps it follows to produce this output.

  2. Data: Describe the dataset that you selected in terms of the attributes present in the data, the number of instances, missing values, and other relevant characteristics.

     Provide a detail description of the preprocessing of your data. Justify the preprocessing you apply and why the resulting data is the appropriate one for mining clusters from it.

  3. Experiments: For each experiment you ran describe:
     • Data: What data did you use to construct and test clusters? Show the 1st page of the .arff file used to run the experiment.
     • Results: What were the resulting clusters? Does each of the clusters correspond to any meaningful concept?

  4. Summary of Results
     • What was the most meaningful clustering you obtained?
     • Detailed comparison of the results from this project with results from previous projects, in particular project 2.
     • Strengths and the weaknesses of your system.

• Oral Report. We will discuss the results from the individual projects during the class on April 13th 2004. Be ready to show your results and to discuss your project in 7 minutes during class. PREPARE SLIDES SHOWING YOUR WORK.