AIRG Topics - Spring 2007

Our group meets on Thursdays at 11:00 a.m., FL 246.

Dates and topics for this semester are as follows:

Jan 18

{meeting postponed}

Jan 25

AIRG Organizational Meeting

Feb 1

No meeting

Feb 8

No meeting

Feb 15

Academic Advising Appointment Day: No meeting

Feb 22

No meeting

Mar 1

tbd

Mar 8

Zach Pardos

Thesis progress report

"The Effect of Model Granularity on Student Performance Prediction Using Bayesian Networks"

A standing question in the field of Intelligent Tutoring Systems and User Modeling in general is what is the appropriate level of model granularity (how many skills to model) and how is that granularity derived? This research will explore varying levels of skill generality and measure the accuracy of these models by predicting student performance within the tutoring system called ASSISTment as well as student performance on the Massachusetts standardized state test. Predicting students' state test scores will serve as a particularly stringent real-world test of the utility of fine-grained modeling. The use of Bayes nets is employed to model user knowledge and for prediction of student responses. The ASSISTment online tutoring system was used by over 600 students during the school year 2004-2005 with each student using the system 1-2 times per month through out the year. Each student answered over 100 state test based items and was tutored by the system when they got a state test item incorrect. The models that will be evaluated are skill models of grain sizes containing 1, 5, 39 and 106 skills. The results of this research show that the finer the granularity of the skill model, the better student performance for the online data can be predicted. However, for the standardized test data, it was the 39 skill model that performed the best. The results support the use of fine-grained models in student knowledge assesment and prediction.

Mar 15

Mingyu Feng

Dissertation progress report

"Addressing the Assessing Challenge with the ASSISTment System"

Secondary teachers across the country are being asked to use formative assessment data to inform their classroom instruction. But the negative side of assessment lies in that every hour spent assessing students is an hour lost from instruction. To help solving this dilemma, we have integrated assistance and assessment in the ASSISTment system. The learning results were presented in Razzaq et al. 2005; Razzaq & Heffernan, 2006, 2007, while in this work, we are engaged in addressing the assessing challenge in the system. Our effort has been devoted to achieve more accurate assessment of students. performance using ASSISTment data. In this talk, I will review the past work. Emphasis will be given to our recent work on using mixed-effects modeling to compare different grain-sized skill models and our preliminary result of incorporating item response theory parameters into our prediction model.

Mar 22

Leena Razzaq

Dissertation progress report

"The effect of math proficiency and the level of feedback on learning"

In a Spring 2006 experiment we showed that scaffolding led to higher averages on a middle school mathematics post-test, although the results were not statistically significant. However, when we looked at scores on particular post-test items that students had seen as pretest items, we saw significant differences. For a pretest item which concerned finding the y-intercept from an equation, the ANOVA showed a statistically significant p-value of 0.005 with an effect size of 0.85 on performance on the post-test. This item on finding the y-intercept from an equation proved to be a difficult problem for all of the students and scaffolding helped significantly. We speculated that the scaffolding had a greater positive effect on learning for this item because it was much more difficult for the students than the other items. We thought that this result warranted a closer look at the link between the difficulty of an item and the effectiveness of scaffolding. In this paper, we report on an experiment that examines the effect of math proficiency and the level of feedback on learning.

Mar 29

*Presentation postponed*

Igor Ushakov

Thesis presentation

Title tbd

Apr 5

Jimmy Schementi

Beckett Conference Room, Fuller Labs

*Presentation delayed to:

Wed April 11, 2007 - 11am

"Converting existing web applications to Ruby on Rails"

*and*

Thu April 12, 2007 - 11am

"Productizing Research"

Computer Science research projects are long-term endeavors that are almost entirely represented by software. However, transforming that software into something both suitable for users and easily maintainable can be as challenging as the actual research, and near impossible if the project is already in use. The current Assistment system (version 2.0) is in production use today in Worcester schools, but very difficult to maintain and build upon. This project, Assistment 3.0, is a total conversion of the Assistment system, retaining only features, to ensure its stability, maintainability, architecture, all to produce a better experience for the users. Additional features will also be added as time permits.

Apr 12

Kevin A. Dill, AK 116, 11am, IMGD Seminar Series presentation

"Embracing Emergent Behavior with Goal-Based AI"

The majority of computer games currently use either scripting or state machines for their high-level AI architecture. While these are both powerful techniques, they are what I like to think of as "declarative AI" because they leave the bulk of the intelligence in the hands of the designer, who specifies exactly what the AI should do in any particular situation. Goal-based AI is an alternative architecture which has been used in a number of successful games, including four of the five titles I have worked on personally. In contrast to the techniques listed above, I would call it "decisive AI." Rather than telling the AI what to do, the role of the designer is to tell the AI what factors to consider and how to weigh them, and the AI will then examine the current state when deciding which actions to take next. In this talk I will discuss the strengths and weaknesses of goal-based AI in comparison to scripting and state machines.

Apr 19

Nick Lloyd

MS Thesis presentation

"Measuring Student Engagement in an Intelligent Tutoring System"

Detection and prevention of off-task student behavior in an Intelligent Tutoring System (ITS) has gained a significant amount of attention in recent years. Previous work in these areas have shown some success and improvement. However, the research has largely ignored the incorporation of the expert on student behavior in the classroom: the teacher. Our research re-evaluates the subjects of off-task behavior detection and prevention by developing metrics for student engagement in an ITS using teacher observations of student behavior in the classroom. For off-task prevention we developed a visual reporting tool that displays a representation of a student's activity in an ITS as they progress and gives a valuable immediate report for the instructor.

Apr 26

Stuart Floyd

MS Thesis presentation

"Data Mining Techniques for Prognosis in Pancreatic Cancer"

This thesis focuses on the use of data mining techniques to investigate the expected survival time of patients with pancreatic cancer. Clinical patient data have been useful in showing overall population trends in patient treatment and outcomes. Models built on patient level data also have the potential to yield insights into the best course of treatment and the long-term outlook for individual patients. Within the medical community, logistic regression has traditionally been chosen for building predictive models in terms of explanatory variables or features. Our research demonstrates that the use of machine learning algorithms for both feature selection and prediction can significantly increase the accuracy of models of patient survival. We have evaluated the use of Artificial Neural Networks, Bayesian Networks, and Support Vector Machines. We have demonstrated (P<.05) that data mining techniques are capable of improved prognostic predictions of pancreatic cancer patient survival as compared with logistic regression alone.