Real Instructors Don't Go to Chat Rooms

Karen A. Lemone
Computer Science Department, WPI
100 Institute Road, Worcester, MA 01609 USA
http://www.cs.wpi.edu/
kal@cs.wpi.edu
Abstract:  Using the Web to enhance or as the sole medium for teaching has produced a new range of problems and opportunities. Tools that facilitate such teaching are appearing and enhancing the delivery and activities of these courses. Studies have been made on how readers (usually students) use such material and tools. Systems have been developed that analyze the student profile, adapting both the pages and the navigation. Tools such as chat rooms, bulletin boards, and cooperative learning techniques have all been designed, implemented. used and studied from the student perspective. This paper explores the other side of these issues: the perspective from the author (usually an instructor). ReCourse, the Retargetable Course Generation system, has had the instructor as a focus from inception. Here, we report on what we have discovered, what we have done as a result of these studies, and how these changes have improved and will continue to improve the system for students as well.

Keywords:  authoring, adaptive hypertext, dynamic hypertext, web-based course management systems, distance education

1. Introduction

WebReCourse [5], the Retargetable Course Generator enables creation and reuse of Web courses. It is a secure software system for online course management, allowing instructors to increase the accessibility of online course material and to create a convenient means of communication between instructors and students. It provides many of the enhancements missing in the World Wide Web implementation of hypertext.

ReCourse allows an author to write a single document which can dynamically choose its content when a reader selects a particular topic.

We call this Retargeting, because the information is retargeted to different readers and different audiences. Such retargeting is a staple among course instructors who teach similar courses to different audiences, or teach the same course multiple times under different circumstances.

Previous papers [6,7] have analyzed student use and response to these tools.

Despite what appears to be an overwhelmingly positive response by students, tools such as ReCourse have not become commonplace in the web-enhanced classroom.. In our experience, instructors either shy away from even trying it, use such tools once and not again (thus, for us, defeating the whole Retargeting philosophy!) or use such tools in such a minimal way as to have little impact on the students.

For us, this was particularly frustrating because of all the similar systems, ReCourse was designed to enhance instructor use. The student tools are fairly standard.

Since we knew these tools are viewed positively by students [6,7], we decided to find out why instructors are not as enthusiastic, and to discover what we can do to encourage them to use web tools to enhance their classes. In particular, we wanted to discover what changes need to be made for effective use. We generalize this to the problem of authoring and using integrated tools for World Wide Web courses.

In the sections that follow, we descibe the system first from the student perspective (section 2), and then from the instructor/author perspective (section 3). Both these sections are brief. Further details can be found in [6,7]. In section 4, we briefly present previously published results from student analyses. In section 5, we characterize instructor/authors and present the issues we have discovered from them. From these, we present some solutions that we have implemented into ReCourse and record the effect it has had on instructor use. Because one of the results is a reluctance of author/instructors to respond, we have devised new ways to procure this information. Section 6 summarizes and presents conclusions and some suggestions for future work.

2. ReCourse: the Student Perspective

Systems have been developed [1,2,3,4,9] which analyze the student profile and present appropriate material, that remember where the student was and direct him or her to an appropriate route on reentry to the hypertext, and that save quiz results so that questions that are asked on a later quiz reinforce shallowly understood concepts. ReCourse requires the student to login with a username and password. (Students who just want to see the pages, and don't plan on using the tools can bypass this login.) Once entered, the student sees the course web pages and icons representing the tools. Although the instructor can turn off any of the tools, the available tools are:

  1. Add New User: Allows students to add themselves. For collaborative learning activities, they can specify a group they belong to. This a recent addition to ReCourse, added as a result of instructor feedback.

  2. Home Page Link: no matter where the student is in the hypertext, this links returns him or her to the first (home) page.

  3. Chat Room: This is a standard chat room, and is quite useful for synchrounous communication. We have held office hours in the chat room for these types of courses. The chat room tends not to be quite as useful for more traditional courses where instructors are seeing one another on a regular basis.

  4. Search Tool: This was the first tool added to the basic system as a result of student feedback. With a course consisting on many nodes, (ours have over 100), finding topics even if they have been previously located is a major problem. The search tool solves this problem in the same way that search engines find information. The same drawback exists that some appropriate keyword muct be known. This has not been an issue.

  5. Grade Access: This is the most used tool, as students login to see their current grades and their average. It computes missing grades as 100%, thus telling students what the maximum avarage can ever be. Students like this (Instructors are not quite so sure about this feature).

  6. Site Map: Generated semi-statically (see description in instructor list below), this gives the students a clickable birds-eye view of the hypertext nodes and links.

  7. Bulletin Board: This is the second most used tool although, once again, the instructor influences its use and success.

  8. Quiz Tool: Allows students to take objective "quizzes". Feedback with correct answers and optional discussion is instantaneous. Features include sending the results via email to the instructor, and the ability of the student to leave and resume where he or she left off.

  9. Password Tool: Allows students to change their password. This is usually done the first time they login to change the randomly generated password.

  10. Classlist: Generated form the user entry tool. Creates a list of students and their email addresses as well as a group distribution list.

  11. Help: Explains how to use every tool. Rarely used!

For further details on these tools, see [5].

3. ReCourse: the Instructor Perspective

Instructors have their own versions of the Student Tools as well as some additional tools. Below, we describe only the additonal tools or the instructor features for tools available also to students.

  1. Site Map: The site map is semi-static; that is, instructors can generate a new site map when they wish. We chose this over dynamic because we felt that it would be too slow for sites with a large number of nodes, and because instructors may change the content of a node, but adding and deleting nodes is rarer.

  2. Global Replace: Allows instructors to change entries common to many of the pages. The pages with the source string are listed and the instructor can choose whether to make the change or not on a node by node basis. We find this especially useful for changing the date and semester information when a course is retaught.

  3. ToolBars: Allows the instructor to deselect particular tools as they see fit.

  4. User Info: Allows the instructor to enter, modify and delete student information. Originally, the instructor (or the poor beleagured TA) enterd all students, but this proved prohibitive with large classes.

  5. Grades: The Grade sheet is created automatically from the Users list. The instructor enters the name, number and weight of assignments. As the course proceeds, this information can be modified, and the instructor enters the grades.

  6. Quiz Generation: Allows the instructor to create the quizzes that the students can then take. We use this extensively for homeworks.

  7. Group Mail: Allows the instructor in collaborative learning environments to communicate with the various groups.

4. Student Assessments

In Experiences in Virtual Teaching [6], we presented results from a number of versions of teaching two different virtual courses. This was with a preliminary version of ReCourse. In Assessment of Tools for Virtual teaching [7], we fine-tuned both the survey and the tools. But results were clear. Students like the tools and believe they enhance their learning. There was a slightly more positive response from computer science students than from non-computer science students.

5. Instructor Assessments

Because we had discovered how useful the tools were for us as instructors, we only waited until we felt the tools were robust enough before making them available to other instructors. A ReCourse "team" consisting of both undergraduates and graduate students prepared to monitor their use. We began with a few carefully selected instructors within the Computer Science Department.

Ultimately, this became two instructors, one teaching an undergraduate course and one teaching a graduate course.

The instructor teaching the graduate course found that entering the class members by hand was quite tedious - he had no TA. Group observation showed that he did use the Grading Sheet, but that there were no postings other than our own on the bulletin board, and no indication that the chat room was ever used. The instructor teaching the undergraduate course had a TA, so only the TA complained about entering the students by hand. Nevertheless, we realized that this must be changed and we changed the ReCourse User function to allow students to enter themselves.

The following term we went outside the computer science department and "invited" a few instructors who, we felt, had positive attitudes towards teaching to try ReCourse. One was reasonably computer literate (he had created his own web pages) while the other found computers generally frustrating and had had her pages created by a school web master who does this as a service to faculty members. We also assigned one member of the ReCourse team to work with each instructor.

Because we had not prepared the same level of questionnaire for instructors as for students, we relied on their feedback both to me and to the ReCourse team member. We are currently testing a new group and changing our questionnaire as well.

We have some clear results, however. We characterize them below:

  1. Faculty Status (!) When the ReCourse team (students) sent an email invitation to faculty, inviting them to try out and then use the system, there was virtually zero response. When the author (faculty) sent the same message, many people responded. Still, a few people did not even reply to the faculty email, including faculty who are familiar to the author. Nevertheless, it is clear that faculty tend not to respond to student mail.

  2. Faculty Chaos & Inertia Of those faculty who did respond and declined to use the system, no one had time to even look at it. All mentioned wanting to use it "in the future." All appeared overworked - one mentioned being overwhelmed that term. Even informing them that ReCourse decreases their work and that it is extremely easy to use, failed to enlist them. The ReCourse group is currently discussing techniques to convince these faculty that it causes less not more work and enhances their course (i.e., better course evaluations).

  3. Missing functionality One faculty used it for one term and did not choose to use it again. She wanted a feature not yet available (the grading system does not operate with full spreadsheet features yet). In truth, she had not used either the bulletin board or any of the other tools besides the grading system. The ReCourse group is addressing the issues of users not familar with many software systems, and not comfortable teaching from the web even when they have web pages.

  4. No instructor help One factor is that some instructors do not have teaching assistants to help them.

  5. Unknown One other faculty member responded to our request, said he would like to use it, but, even with urging and help from the ReCourse team, failed to use the tools. He did not respond to our invitation the following term. We continue to solicit response from such users. We believe they may find it difficult to give constructive criticism to other faculty members.

One user used it previously and has returned to use it again. Like us, she has found that it enhances her course and saves her and her ta's work.

We have also enlisted one new user who appears enthusiastic.

6. Author Characteristics

Although we continue to gather information and statistics, we know the following:

  1. The more computer literate an instructor/author is, the more apt they are to use the system, and the more apt they are to use more of the available tools.

  2. The more helpful the assigned ReCourse team member is, the more apt the author/instructor is to use the tool in its entirety.

  3. Newer faculty are much more apt to respond to the invitation, to use more of the tools, and to return to use it again. This is due to more experienced instructors having (existing, but not as appropriate) tools they are used to using and the general chaos that surrounds faculty members as they find themselves with more service activities. The ReCourse team believes as they hear that the tool is an asset, they will make the (small) effort to use it.

7. Conclusions

ReCourse is documentably popular and useful for web courses. Students like it. But students have grown up in a computer world and, now, even in a web world. Instructor/authors have more inertia. We are continuing to gather statistics from both the users and non-users in an attempt to encourage use of a system which improves the delivery of web courses. It is likely that a new type of education is needed to create the appropriate mindset for using tools for web-enhanced courses.

References

  1. Brusilovsky P. (1996) Methods and techniques of adaptive hypermedia. In User Modeling and User-Adapted Interaction 6, 2-3, 87-129.
  2. Clibbon K. (1995) Conceptually adapted hypertext for learning. In I.Katz, R. Mack, & L. Marks (Eds.), Proceedings of CHI'95 (pp.224-225). Denver: ACM. http://www.acm.org/sigchi/chi95/Electronic/documnts/shortppr/kc_bdy.htm.
  3. Calvi, Licia & De Bra, P. (1998), Towards a Generic Adaptive Hypermedia System, University of Antwerp, Belgium, In Second Workshop on Adaptive Hypertext and Hypermedia, Pittsburgh, USA.
  4. Eklund J & Sawers J (1996) Customising Web-based course delivery in WEST with navigation support. Proceedings of WebNet'96, World Conference of the Web Society. San Francisco, CA, October 15-19, 1996, pp. 534-535.
  5. Lemone, K. (1996).  Retargetable Course Generation: a Method for Reusability in Distance Education   In Workshop on Architectures for Intelligent Tutoring Systems, Montreal, Canada.
  6. Lemone, K. (1997).  Assessment of Tools for Virtual Teaching   In Proceedings of Enable '97, Espoo, Finland.
  7. Lemone, K. (1997).  Experiences in Virtual Teaching  In Proceedings of WebNet'97, Toronto, Canada.
  8. Lemone, K. (1998) Issues in Authoring Adaptive Hypertext on the Web, Second Workshop on Adaptive Hypertext and Hypermedia, Pittsburgh.
  9. Zeiliger R, Reggers T & Peeters R (1996) Concept-map based navigation in educational hypermedia : a case study. In Proceedings of ED-MEDIA'96 - World conference on educational multimedia andy hypermedia. Boston, MA.