WPI Worcester Polytechnic Institute

Computer Science Department

AIRG Topics - Spring 2010

Our group meets on Thursdays at 11:00 a.m., FL 246.

Jan 28
Yue Gong
Comparing Knowledge Tracing and Performance Factor Analysis by Using Multiple Model Fitting
    Student modeling is very important for ITS due to its ability to make inferences about latent student attributes. Although knowledge tracing (KT) is a well-established technique, the approach used to fit the model is still a major issue as different model-fitting approaches lead to different parameter estimates. Performance Factor Analysis, a competing approach, predicts student performance based on the item difficulty and student historical performances. In this study, we compared these two models in terms of their predictive accuracy and parameter plausibility. For the knowledge tracing model, we also examined different model fitting algorithms: Expectation Maximization (EM) and Brute Force (BF). Our results showed KT+EM is better than KT+BF and comparable with PFA in predictive accuracy. We also examined whether the models’ estimated parameter values were plausible. We found that by tweaking PFA, we were able to obtain more plausible parameters than with KT.

Feb 4
Dovan Rai
Mily's World: A Coordinate Geometry Learning Environment with Game-like Properties
    Mily's World is a learning environment for coordinate geometry that has game-like properties, that is, elements of games that are engaging such as cover story, graphical representation, and animated feedback. This paper proposes that adding game-like properties to a computer tutor results in more student engagement and interest in the material. However, in addition to taking instructional time away, adding such properties imposes new limitations and difficulties in constructing content. Therefore, we have taken a measured and minimalist approach to making the original environment more game-like by weighing each additional component in terms of retaining all the learning features of a tutor and minimizing the new limitations, while exploiting the benefits of games. Sixty six students used the system who had also used Assistment, a web based tutor. Although the students were not totally enthusiastic about Mily, they still preferred it over Assistment. We also analyzed how different student subpopulations receive the new intervention, and that found that the students who find real-world examples and pictures helpful for solving math reported liking Mily more.

Feb 11
Elijah Forbes-Summers
Classifying movies based on user ratings using HLDA
    This work describes recovering genre information (and other salient groupings) for movies based on user ratings in the netflix prize dataset. The goal of the project was to become more familiar with a particular type of model, Hierarchical Latent Dirichlet Allocation. I will discuss the model and its relation to other Bayesian models and discuss the results of its application to the netflix dataset.

Feb 18
Academic Advising Appointment Day: No meeting

Feb 25
Zach Pardos
Navigating the parameter space of Bayesian Knowledge Tracing models: Visualizations of the convergence of the Expectation Maximization algorithm.
    Bayesian Knowledge Tracing (KT) models are employed by the cognitive tutors in order to determine student knowledge based on four parameters: learn rate, prior, guess and slip. A commonly used algorithm for learning these parameter values from data is the Expectation Maximization (EM) algorithm. Past work, however, has shown that with four free parameters the standard KT model is prone to converging to erroneous degenerate states depending on the initialized values of these four parameters. In this work we simulate data from a model with known parameter values and then brute force the parameter initialization space of KT to map out which values lead to erroneous learned parameters. Through analysis of multi-dimensional visualizations we found that the initial parameter values leading to a degenerate state are not scattered randomly throughput the parameter space but instead exist within predictable boundaries. A recently introduced extension to KT that individualizes the prior parameter is also explored and compared to standard KT with regard to parameter convergence. We found that the individualization model has a unique property which allow for a more informed selection of initial parameters.

March 4
Andreea Bodnari
Mapping Text to Knowledge using Natural Language Processing
    In this talk we describe our design and implementation of a system that analyzes text corpora. This system uses natural language processing techniques to extract knowledge from written text and represents this knowledge as a network. The system displays this network to the user and allows the user to interactively explore the network. Possible applications are in social networks, and text simplification.

Mar 11

Mar 18
Matt Dailey
Learning what works in an ITS from non-traditional randomized controlled trial data
    Traditionally in ITS, researchers run a randomized controlled trial (RCT) using a standard pretest posttest design to find out which items or tutorial help cause more learning. However, in practice, ITS creators make content without knowing ahead of time which questions or tutorial help work best, without wanting to spend the extra time to create experiments and possibly without identifying possible experiments in which to embed their content. As a result, there is much data that we would like to analyze, but may not be. In this presentation we discuss empirical validity and applications of the Item Effect Model, a Bayesian networks method introduced by Pardos and Heffernan to determine which item is most effective for learning amongst a set of items of the same skill when all the items are given in random order. We apply this model to 11 data sets from a mathematical web based tutoring system. We first manipulate problem sets which were not designed as RCT, but which took on the standard RCT form. We then apply this model to a different domain, by analyzing the effectiveness of different tutorial feedback strategies embedded in Mastery Learning problem sets. The validity of our model is shown through both subject matter experts, and comparison to more traditional statistical methods. We found that the tutorial help or item chosen by the Bayesian method as having the highest rate of learning agreed with the traditional analysis in 9 out of 11 of the experiments. The practical impact of this work is an abundance of knowledge about what works that can now be learned from the thousands of experimental designs intrinsic in datasets of tutoring systems that assign items in a random order.

Mar 25
Matt Bachmann

Apr 1
Special CS Faculty Mtg: no AIRG

Apr 08
Adam Goldstein
Detecting the Moment of Learning
    Intelligent Tutoring Systems have become increasingly accurate at detecting whether a student knows a skill at a given time. However, these models do not tell us exactly at which point that skill is learned. We will present a machine-learned model P(J) that can assess the probability that a student JustLearned a skill at a specific problem step. A method that can predict such information with a relatively high level of accuracy can potentially provide a means for individualized support and feedback within the system. We will show the effectiveness of our model within the Cognitive Tutor and present our progress using subsequent action steps in the ASSISTment tutoring system. The talk will include an analysis of gradual and "eureka" style learning of math skills through a discussion of spikiness of P(J) and will explore means to improve our results.

Apr 15
Jonathon Gibbons talk postponed
There is an AI-related talk at 11am in the IMGD speaker series, by
Damian Isla
"Next-Gen Content Creation for Next-Gen AI"
Fuller Labs, Lower Perrault Hall

Apr 22
Undergraduate Project Presentation Day
Jeff Moffett
Applying Causal Models to Dynamic Difficulty Adjustment in Video Games
    We have developed a causal model of how various aspects of a computer game influence how much a player enjoys the experience, as well as how long they will play. This model is organized into three layers: a generic layer that applies to any game, a refinement layer for a particular game genre, and an instantiation layer for a specific game. Two experiments using different games were performed to validate the model. The model was used to design and implement a system and API for Dynamic Difficulty Adjustment. This API uses machine learning techniques to make changes to a game in real time in the hopes of improving the experience of the user and making them play longer. A final experiment is presented that shows the effectiveness of the designed system.

Apr 29
Graduate AI class presentations

May 6
Elijah Forbes-Summers
    There have been many modifications made to knowledge training attempting to increase the accuracy of its predictions or expand the scope of phenomena it models. Here's mine. I added information about how fast students respond to questions and evaluated the results on data from the ASSISTments system.

[Feedback] [Search Our Web] [Help & Index]

[Return to the WPI Homepage] [Return to the CS Homepage]

AIRG Coordinator / Thu May 6 14:19:50 EDT 2010