Mingyu Feng Ph.D. Dissertation, and Ph.D. Dissertation Defense Presentation Department of Computer Science Worcester Polytechnic Institute E-Mail: mfeng 'at' cs.wpi.edu

I am now a Research Scientist at Center for Technology (CTL) in Learning at SRI International .

---

Biography

I graduated with a Ph.D. in Computer Science from Department of Computer Science at Worcester Polytechnic Institute in Aug., 2009. My advisor is Prof. Neil T. Heffernan. I received my B.S. in Computer Science from Tianjin University in 1999, and M.S. degree in the same field from Tianjin University in 2002. I joined Department of Computer Science at WPI for a Ph.D. in 2004. Before that, I was a software engineer in Beijing, China.

My research goal is to create educational technologies that dramatically increase student achievement. Towards this goal, currently my primary interests lie in the areas of intelligent tutoring systems, particularly, student modeling and educational data mining. I have also worked in the area of cognitive modeling, and psychometrics as well. I am most interested in user modeling and innovations in assessment, and related computing technologies. The title of my Ph.D. dissertation is Towards assessing students' fine grained knowledge: Using an intelligent tutor for assessing.

I have been a primary member of the ASSISTment project. We developed a web-based tutoring system that assesses at the same time. After analyzing 1000 students' data of using the system, I developed metrics measures response efficiency that accounts for the trials it takes students to come up with an answer to a problem, the time they take to correct an answer if it is wrong, help-seeking behavior (e.g. the number of hints they request), and their performance on the sub-steps (called scaffolding questions). My results show by taking into consideration student-system interaction information that has been ignored by traditional assessment approaches, we can reliably improve the prediction accuracy of student proficiency. This work was well received and was nominated for best student paper award at the International Conference on World Wide Web. And the editor from User Modeling and User-Adapted Interaction (UMUAI) journal mentioned this is UMUAI's first "accept pending minor revisions" in quite a few years.

I am used to work in multicultural environments. Due the interdisciplinary nature of my research, I have strong collaboration with researchers from Computer Science, Psychology (education psychology and cognitive psychology), Statistics, and Psychometrics. I have also worked with subject-matter experts and school teachers.

My research has contributed to the design and evaluation of educational software, has developed computing techniques to address problems in user learning, and has produced basic results on the tracking student learning of mathematical skills. I have authored over 20 peer-reviewed publications, including 11 conference papers, 1 book chapter, and 8 journal papers, with another 2 papers in submission.

---

Education:

• Ph.D. in Computer Science. Worcester Polytechnic Institute, Worcester, MA. Aug., 2009 (Exp.). GPA: 4.0
  • Ph.D. dissertation committee:
        Neil T. Heffernan, Associate Professor, Worcester Polytechnic Institute
        Carolina Ruiz, Associate Professor, Worcester Polytechnic Institute
        Joseph Beck, Research Scientist, Worcester Polytechnic Institute
        Kenneth R. Koedinger, Professor, Carnegie Mellon University

• M.S. in Computer Science, Tianjin University, Tianjin, China. May, 2002. GPA: 85/100
• B.S. in Computer Science, Tianjin University, Tianjin, China, English minor. May, 1999. GPA: 88/100.

Research interests:

• Intelligent Tutoring Systems;
• Machine learning and data mining in learning sytems;
• User modeling
• Learner assessment and related computing technologies;
• Human Computer Interaction

Research and development experiences:

• Summer research intern.                 June, 1st - August 1st, 2008
  Research and Development, Education Testing Service (ETS), Princeton, NJ
  Mentor: Dr. Eric Hansen
  Worked on the project of Pilot testing of Evidence Centered Design (ECD) for learning-oriented products. Reviewed and helped enhancing the framework of ECD for learning and used it to examine a learning system. (Related publication: #1, #20).

• Research Assistant                    Jan 2004 - present
  Department of Computer Science, Worcester Polytechnic Institute
  Primary member of the ASSISTment project (http://www.assistment.org). We developed a web-based tutoring system to help middle and high school students to prepare for a high-stake standardized state test required by No Child Left Behind act. The system offer instruction to students while providing a detailed evaluation of their abilities to the teachers. The system is currently being used regularly by more 3000 students from Worcester Public Schools, MA.

• Software Engineer                     Oct., 2002 - Aug., 2003
  Beijing Si-Tech Information Technology Co. , Beijing, China
  Development of Information On Demand System 2.0, a system that provides text message services to cell phone users. The system was implemented on SunOS, by Sybase ASE, C language plus Java and Java Script, J2EE.

• Research Assistant                     Sept. 1999 - May, 2002
  Dept. of Computer Science, Tianjin University
  Primary member of National 863 High Technology Paradigm Project (team project of over 20 people), cooperated with Kelon Corp. We developed a sales & marketing information management system. The system was based on Browser/Server computing architecture, distributed Database (Oracle), and national-scale Intranet. (related publication: #16, #17)

• Teaching Assistant                     Sept. 2001 - May, 2002
  Dept. of Computer Science, Tianjin University
  Teaching assistant of graduate course Object-Oriented Methodology

Research Projects:

• Reliable math proficiency assessment using intelligent tutoring systems
  

  Dynamic testing- Traditional tests only pay attention to whether a student's response to a question is correct or wrong. I paid attention to consideration student-computer interaction data in intelligent tutoring systems and developed metrics to measure the amount of assistance a student needs to solve a problem, and how much time a student requires to finish a tutoring session. Our approach can assess student more accurately than traditional method, and showed meanwhile students were learning from the system. (related publication: J6, CP5, CP3, CP2, WP2)

  Longitudinal data analysis - Student knowledge is not static but developed over time. In order to better understand the learning process and thus better assess student performance, I investigated student knowledge development longitudinally using mixed-effects modeling approach. (related publication: JS2, CP4, CP2, PP1, WP3).

  Cognitive assessment - On top of overall performance estimate, both instructors and students want detailed reports to inform their instruction and learning. We constructed different grain sized cognitive models and developed skill-level reports. Additionally, I analyzed item level, longitudinal data and found the finer grained model can be used to better predict students' end-of-year exam scores. (related publication: J5, JS2, CP4, PP1, WP3)

  Refine cognitive model through learning factor analysis (in progress) - Creating an accurate model of a students' knowledge can be quite difficult due to various sources of uncertainty. The first model is the best guess and should be iteratively refined after usage in intelligent tutoring systems. In addition to providing data analysis results to subject matter experts so that they can manually improve the existing models, I am working on refine existing models using Learning Factor Analysis (LFA), a method involving artificial intelligent searching, and statistical modeling.

• Effectiveness of tutoring within the intelligent tutoring system
  

  I designed, carried out and analyzed experiments to detect if and how much students are learning from ASSISTment. I also applied educational data mining approaches to compare effectiveness of different instructional content, and thus demonstrated an easier and quicker approach of evaluating the quality of contents in a learning system other than experimental studies (related publication: CP8, CP6)

• Design and implement reporting system in the ASSISTment project
  

  I designed and implemented the first web-based reporting system in the ASSISTment project in 2004. It provided real-time performance evaluation reports to teachers while students were working. The system also allowed researchers to set up randomized controlled experiments. The reporting system performed experiment analysis and delivered the results automatically. (related publication: J4, J3, WP1)

Publications:

Book chapters

• B2.  Feng, M., Heffernan, N.T., & Koedinger, K.R. (2010). Student modeling in an Intelligent Tutoring System. In Stankov, Glavinc, and Rosic. (Eds.) Intelligent Tutoring Systems in E-learning Environments: Design, Implementation and Evaluation. pp. 208-236. Hershey, PA: Information Science Reference. (pdf)

• B1.  Razzaq, L., Feng, M., Heffernan, N., Koedinger, K., Nuzzo-Jones, G., Junker, B., Macasek, M., Rasmussen, K., Turner, T., & Walonoski, J. (2007). Blending Assessment and Instructional Assistance. In Nedjah, Mourelle, Borges and Almeida (Eds). Intelligent Educational Machines within the Intelligent Systems Engineering Book Series . pp.23-49. Springer Berlin / Heidelberg. (pdf)

Journal papers

• J8.  Razzaq, L., Parvarczki, J., Almeida, S.F., Vartak, M., Feng, M., Heffernan, N.T. and Koedinger, K. (2009). The ASSISTment builder: Supporting the Life-cycle of ITS Content Creation.  IEEE Transactions on Learning Technologies, 07 May. 2009. IEEE computer Society Digital Library. IEEE Computer Society, <http://doi.ieeecomputersociety.org/10.1109/TLT.2009.23>

• J7. Feng, M, Heffernan, N., Heffernan, C. & Mani, M. (2009). Using mixed-effects modeling to analyze different grain-sized skill models in an Intelligent Tutoring System. IEEE Transactions on Learning Technologies, vol. 2, no. 2, pp. 79-92, Apr.-June 2009, doi:10.1109/TLT.2009.17. (pdf) (Based on WP3)( Featured article of the issue; was also selected from all TLT papers published in 2009 to represent IEEE TLT in the 2009-2010 IEEE Computer Society Publications Sampler.)

• J6.  Feng, M., Heffernan, N.T., & Koedinger, K.R. (2009). Addressing the assessment challenge in an Online System that tutors as it assesses. In User Modeling and User-Adapted Interaction: The Journal of Personalization Research (UMUAI journal). 19(3), 243-266, August, 2009. (pdf)  (Based on CP2) ( Winner of 2009 James Chen Annual Award - Best UMUAI paper of the year ) (This paper is discussed in the National Education Technology Plan that just came out in March 2010. This paper is cited for how it shows that you can better assess student if you pay attention to how much assistance students need. This paper is important as psychometricians normally think that the best was to assess students is to not give students any feedback during the test assumping that it is harder to measures a moving target (if kids are allowed to learn during the test, than their knowledge level is a "moving target"). Counter-intuitively, this paper shows is that you can better assess student knowledge because can use the addition information of how many hints they needed to solve the problem, how many attempts they made, and how much time they needed to answer. This result has practical significance as it means that schools can do less testing where kids don't get feedback, and yet still do valid assessment. In fact *better* assessment. This paper also pointed out that an area of future work would be to see if this result holds up even if time on task is held constant. I has subsequently done a study that showed evidence to suggest that if you control for time on task, you still get better assessment of student knowledge, even though they do fewer questions if they spend the time to learn how to solve the problem. See the follow-up paper CP10 for details on this study. Recently, this paper (J6) was also cited in an Education Week article Dec 13, 2010 called "'Data Mining' Gains Traction in Education" by Sarah D. Sparks)

• J5.  Razzaq, L., Heffernan, N., Feng, M., Pardos, Z. (2007). Developing Fine-Grained Transfer Models in the ASSISTment System. Journal of Technology, Instruction, Cognition, and Learning , Vol. 5. Number 3. Old City Publishing, Philadelphia, PA. 2007. pp. 289-304.(pdf)  (Based on WP3)

• J4.  Feng, M. & Heffernan, N. (2007). Towards Live Informing and Automatic Analyzing of Student Learning: Reporting in the Assistment System. Journal of Interactive Learning Research. 18 (2), pp. 207-230. Chesapeake, VA: AACE. (pdf)  (Based on J3, WP1)

• J3.  Feng, M., Heffernan, N.T. (2006). Informing Teachers Live about Student Learning: Reporting in the Assistment System. Technology, Instruction, Cognition, and Learning Journal. Vol. 3. Old City Publishing, Philadelphia, PA. 2006. (pdf [preview version]) (Based on WP1)

• J2.  Feng, M., Zhao, Z., & Zhang, G. (2002), Data collision and its solution in distributed situation (in Chinese), Computer Application Research (Chinese kernel academic journal), 2002,19 (2) :72-74.

• J1.  Feng, M., & Zhao, Z. (2002), Research of Reconfigure Information System Based on Workflow and CORBA (in Chinese), Microcomputer and Its Application (Chinese kernel academic journal), 2002, 1, 36-38.

Strictly peer reviewed conference papers (Percent acceptance rate in the 30s or below)

• CP10.  Feng, M., Heffernan, N.T. (2010). Can We Get Better Assessment From a Tutoring System Compared to Traditional Paper Testing? Can We Have Our Cake (Better Assessment) and Eat It Too (Student Leaning During the Test)? To appear in Baker, Merceron & Palvik (Editors) of the Proceedings of the 3rd International Conference on Educational Data Mining (EDM 2010), Pittsburgh, PA. 2010. (pdf)

• CP9.  Feng, M., Beck, J., Heffernan, N.T. (2009). Using Learning Decomposition and Bootstrapping with Randomization to Compare the Impact of Different Educational Interventions on Learning. In Barnes, Desmarais, Romero, & Ventura (Eds.), Proceedings of the 2nd International Conference on Educational Data Mining. pp. 51-60. Cordoba, Spain: Copisterias Don Folio, S.L. (pdf)

• CP8.  Feng, M., Heffernan, N.T., Beck, J. (2009). Using learning decomposition to analyze instructional effectiveness in the ASSISTment system. In Dimitrova, Mizoguchi, du Boulay, and Grasser (Eds), Proceedings of the 14th International Conference on Artificial Intelligence in Education (AIED-2009). pp. 523-530. Amsterdam, Netherlands: IOS Press. (pdf)

• CP7.  Hansen E. G., Zapata-Rivera, D., & Feng, M. (2009). Beyond Accessibility: Evidence Centered Design for Learning (ECDL) for Improving the Efficiency of Instruction. Paper presented at the session of Test use in special populations at the National Council on Educational Measurement 2009 Annual Conference (NCME, 2009), San Diego, CA. pdf

• CP6.  Feng, M., Heffernan, N., Beck, J, & Koedinger, K. (2008) Can we predict which groups of questions students will learn from? In Baker & Beck (Eds.). Proceedings of the 1st International Conference on Education Data Mining. pp.218-225. Montr¨¦al 2008.(pdf)

• CP5.  Feng, M., Beck, J,. Heffernan, N. & Koedinger, K. (2008) Can an Intelligent Tutoring System Predict Math Proficiency as Well as a Standardized Test? In Baker & Beck (Eds.). Proceedings of the 1st International Conference on Education Data Mining. pp.107-116. Montr¨¦al 2008. (pdf) (Based on CP2)

• CP4.  Feng, M., Heffernan, N. T. (2007). Assessing Students. Performance Longitudinally: Item Difficulty Parameter vs. Skill Learning Tracking. Paper presented at the 2007 Annual meeting of National Council of Measurement on Educational (NCME'2007), Chicago. (pdf) (Based on WP3)

• CP3.  Feng, M., Heffernan, N.T, Koedinger, K.R. (2006b). Predicting State Test Scores Better with Intelligent Tutoring Systems: Developing Metrics to Measure Assistance Required. In Ikeda, Ashley & Chan (Eds.). Proceedings of the 8th International Conference on Intelligent Tutoring Systems. Springer-Verlag: Berlin. pp. 31-40. 2006. (pdf)

• CP2.  Feng, M., Heffernan, N.T, Koedinger, K.R. (2006a) Addressing the Testing Challenge with a Web-Based E-Assessment System that Tutors as it Assesses. In Proceedings of the 15th International World Wide Web Conference. pp. 307-316. New York, NY: ACM Press. 2006. (pdf xhtml) (ppt) Best Student Paper Nominee. (acceptance rate: 11%, 75/667; E*Applications Track acceptance rate: 8%, 9/120)

• CP1.  Razzaq, L., Feng, M., Nuzzo-Jones, G., Heffernan, N.T., Koedinger, K. R., Junker, B., Ritter, S., Knight, A., Aniszczyk, C., Choksey, S., Livak, T., Mercado, E., Turner, T.E., Upalekar. R, Walonoski, J.A., Macasek. M.A., Rasmussen, K.P. (2005). The Assistment Project: Blending Assessment and Assisting. In C.K. Looi, G. McCalla, B. Bredeweg, & J. Breuker (Eds.) Proceedings of the 12th International Conference on Artificial Intelligence in Education, pp. 555-562. Amsterdam: ISO Press. (pdf)

Poster papers in pretigious conferences (percent acceptance rate in the 50-60s%)

• PP4.  Feng, M., Heffernan, N.(2010). Can We Get Better Assessment From a Tutoring System Compared to Traditional Paper Testing? Can We Have Our Cake (Better Assessment) and Eat It Too (Student Leaning During the Test)? In Proceedings of the 10th International Conference on Intelligent Tutoring Systems (ITS 2010), Pittsburgh, PA. 2010. (pdf)

• PP3.  Feng, M., Heffernan, N. (2010). Using Data Mining Findings to Aid Searching for Better Skill Models. In Proceedings of the 10th International Conference on Intelligent Tutoring Systems (ITS 2010), Pittsburgh, PA. 2010. (pdf)

• PP2.  Feng, M., Beck, J. (2009). Back to the future: a non-automated method of constructing transfer models. In Barnes & Desmarais (Eds.), Proceedings of the 2nd International Conference on Educational Data Mining. pp. 240-249. Cordoba, Spain: Copisterias Don Folio, S.L. (pdf)

• PP1.  Pardos, Z., Feng, M. & Heffernan, N. T. & Heffernan-Lindquist, C. (2007).Analyzing fine-grained skill models using bayesian and mixed effect methods. In Luckin & Koedinger (Eds.) Proceedings of the 13th Conference on Artificial Intelligence in Education. Amsterdam, Netherlands: IOS Press.pp.626-628. (pdf) (Based on WP3, WP4)

Workshop and less stringently reviewed venues

• WP5.  Feng, M., Hansen, E. & Zapata, D. (2009). Using Evidence Centered Design for Learning (ECDL) to examine ASSISTment. Paper presented at the annual meeting of America Educational Research Association (AERA), San Diego, CA. April, 2009. pdf

• WP4.  Pardos, Z., Feng, M., Heffernan, N. T., Heffernan-Lindquist, C.& Ruiz, C. (2007). Analyzing fine-grained skill models using bayesian and mixed effect methods. In Heiner, C., Heffernan, N., & Barnes, T. (Eds). Educational Data Mining: Supplementary Proceedings of the 13th International Conference of Artificial Intelligence in Education. Marina del Rey, CA. pp. 50-59.

• WP3.  Feng, M., Heffernan, N.T, Mani, M. & Heffernan C. (2006). Using Mixed-Effects Modeling to Compare Different Grain-Sized Skill Models. In Beck, J., Aimeur, E., & Barnes, T. (Eds). Educational Data Mining: Papers from the AAAI Workshop. Menlo Park, CA: AAAI Press. pp. 57-66. (pdf ppt)

• WP2.  Feng, M., Heffernan, N.T., Koedinger, K.R., (2005). Looking for Sources of Error in Predicting Student's Knowledge. In Beck. J. (Eds). Educational Data Mining: Papers from the 2005 AAAI Workshop. Menlo Park, California: AAAI Press. pp. 54-61. (pdf)

• WP1.  Feng, M., Heffernan, N.T. (2005). Informing Teachers Live about Student Learning: Reporting in the Assistment System. The 12th International Conference on Artificial Intelligence in Education Workshop on Usage Analysis in Learning Systems, 2005, Amsterdam. (pdf)

Other articles

• O2.  Hansen, E.G.,Zapata-Rivera,D.,& Feng, M. (in preparation). Pilot testing and enhancing evidence center design for learning oriented products. (Based on WP5)

• O1. Weitz,R., Heffernan, N.T.,Rosenthal, D.& Feng, M. An analysis of middle-school math errors across schools. WPI Technical Report. WPI-CS-TR-08-07.

Awards:

• Winner of 2009 James Chen Annual Award - Best UMUAI paper of the year, 2010

• Student travel grant from the National Science Foundation (NSF) for the 9th International Conference on Intelligent Tutoring Systems (ITS-2008), 2008

• Student travel grant from the National Science Foundation (NSF) for the 13th International Conference on Artificial Intelligence in Education (AIED-2007), 2007

• Nominated for "Best Student Paper" award, the 15th International Conference on World Wide Web (WWW-2006), 2006

• Huawei Outstanding Graduate Student Award (top 1 graduate student), Tianjin University. China. 2002.

• Motorola Outstanding Undergraduate Student Award (top 5 undergraduate student). Tianjin University, 1996, 1997.

• Outstanding Student Prize. Tianjin University. China. Yearly, 1996-2001

Professional Service:

• Reviewer: International Conference of Artificial Intelligence in Education (AI-ED), 2007, 2009; International Conference on Intelligent Tutoring Systems (ITS) 2008, 2010; International Conference on Educational Data Mining (EDM), 2008, 2009; International Conference on Computers in Education (ICCE), 2008; American Educational Research Association (AERA) annual meeting, 2009.

• Program Committee: International Conference on Educational Data Mining, 2010; Workshop on Educational Data Mining as part of International Joint Conference on Artificial Intelligence (IJCAI), 2007.

mfeng at cs.wpi.edu

Last modified: Apr. 22nd, 2010