Some sample datasets (including the ones provided with the Weka System)

• credit_history.arff
• Subset of contact-lenses.arff
• contact-lenses.arff
• cpu.arff
• iris.arff
• labor.arff
• soybean.arff
• weather.arff
• weather.nominal.arff

• Day: Thursday Aug. 26, 2004

• Useful Slides and Class Handout:

  • Witten's and Frank's textbook slides - Chapter 1.

• Lecture Objectives:
  

  Here are the main topics covered in this lecture:

  1. Introduction to Knowledge Discovery in Databases (KDD) and Data Mining

     • Textbook slides provided above. Cover slides 1-18, 21 plus one of the applications described in Slides 22-31.

  2. Course Logistics

     • Distribute course Syllabus

     • Prof. Ruiz is away this week participating in a professional conference. She'll be back on Monday, Aug. 30th. She'll be happy to take care of your requests (signing BS/MS forms, ...) and any other questions you may have for her when she comes back.

     • See the course webpage containing the course syllabus

     • See the course schedule of topics.

     • We'll have 4 homework assignments, 4 projects, and 2 exams in this course.

• Additional Notes:

  • Before our next lecture everybody should:

    • Read Chapter 1 of the Witten's and Frank's textbook

    • Download the Weka system according to the syllabus instructions

    • Play with the Weka system in preparation to tomorrow's lecture

    • Start reading Chapter 2 and Chapter 8 of the textbook to become familiar with the Weka system.

• Day: Friday Aug. 27, 2004

• Useful Slides and Class Handout:

  • Witten's and Frank's textbook slides - Chapter 2.

• Lecture Objectives:
  

  The main objective of this lecture is to become familiar with the Weka system.

  1. Textbook slides provided above. Cover slides 1-14, 26-32.

  2. Work with the Weka System

     1. Get familiar with the sample datasets provided with the Weka system.

     2. Open the contact lenses dataset contact-lenses.arff with a word editor. The file consists of:

        • An OPTIONAL header with general information about the dataset. Each line in this header starts with "%" to denote that these are commented lines.

        • The name of the dataset

            @relation contact-lenses
                                     

        • The name and types of the attributes in the dataset

            @attribute age                  {young, pre-presbyopic, presbyopic}
            @attribute spectacle-prescrip   {myope, hypermetrope}
            @attribute astigmatism          {no, yes}
            @attribute tear-prod-rate       {reduced, normal}
            @attribute contact-lenses       {soft, hard, none}
                                     

          These attributes are the "column names" of the tabular data that comes after.

        • The dataset instances

            @data
          
            young,myope,no,reduced,none
            young,myope,no,normal,soft
            young,myope,yes,reduced,none
            young,myope,yes,normal,hard
             ...
          
                             

  3. Run the Weka system. Choose the "Explorer" option in the main window.

  4. Load the contact lenses dataset into the Weka System using "Open File".

  5. Apply the following classification techniques (under "Classify") to this dataset

     • ZeroR: 0 predicting attributes are used to construct a classification rule

     • OneR: 1 predicting attribute is used to construct classification rules

     • Id3: Decision Trees

  6. If time allows, apply the following association technique (under "Associate") to this dataset

     • Apriori

  7. Go back to the "Preprocess" tab and modify the input dataset using the following preprocessing filters. Access the filters by clicking on the "Choose" button.

     • Remove This filter removes one or more attributes from the current dataset.
       1. Select "Remove" from "weka->filters->unsupervised->attribute"
       2. Click on "Remove" to access the Weka window with the parameters for this filter
       3. Click on "More" to learn more about this filter
       4. Use say "1-2" in the "AttributeIndices"
       5. Click "OK"
       6. Click "Apply" on the main Weka Window
       7. Describe how the dataset changed

     • Resample This filter selects a random sample from the data instances in the current dataset.
       1. Select "Resample" from "weka->filters->unsupervised->instance"
       2. Click on "Resample" to access the Weka window with the parameters for this filter
       3. Click on "More" to learn more about this filter
       4. Use say "20.0" in the "SampleSizePercent" to select a subsample of 20% of the current instances.
       5. Click "OK"
       6. Click "Apply" on the main Weka Window
       7. Describe how the dataset changed

  8. If time permits, discuss how to convert a dataset to ARFF (attribute relation file format). Use for example the Titanic Dataset. Look at the dataset description and the Data instances.

• Additional Notes:

  • Before our next lecture everybody should:

    • Finish reading Chapter 2 of the Witten's and Frank's textbook.

    • Continue playing with the Weka system.

    • Continue reading Chapter 8 of the textbook.

    • Find a project partner.

    • Go over the course syllabus and the course webpages in detail and prepare any question you may have about them for Monday's lecture.

• Day: Monday Aug. 30, 2004

• Useful Slides and Class Handout:

  • Prof. Ruiz's KDD Process Slides (distributed as a handout during class).

• Lecture Objectives:
  

  1. To learn in detail the different steps of the Knowledge Discovery in Databases (KDD) process.

  2. To discuss different approaches to data integration: mediation and data warehousing approaches.

  3. To learn more about KDD and data mining applications.

  4. To learn about some commercial and academic data mining systems.

  5. To learn about additional data mining and KDD resources (books, journals, conferences).

• Additional Notes:

  • Before our next lecture everybody should:

    • Finish reading Chapter 1 and Chapter 2 of the Witten's and Frank's textbook.

    • Read Chapter 3 of the Witten's and Frank's textbook.

    • Choose a project partner (except if you are taking this course for BS/MS credit).