Assessment of Peer Learning in the Introductory Computer Science
Curriculum
Assessing the effect of any change is important. They key idea in
assessment is MINMAXCON---minimize variance, maximize the treatment and control
for interfering variables.
As background for the group, a number of references for qualitative and quantative
assessment were presented by Dr. Jim Groccia at the workshops.
Qualitative (vs. quantatative) assessment seeks to obtain results by
soliciting comments from participants rather than simply collecting
numerically results.
In assessing changes to the classroom approach, Dr. Groccia suggests
telling the class they are part of an experiment and to not to be hesitant
in setting positive expectations. If prior outcome results are available
then these can be shared with the students.
The following outlines types of assessment that can be used for the
evaluation of activities associated with cooperative learning.
In all cases the subjects to be evaluated are the participants in the peer
learning activities. The objective of the assessment can be a specific
activity, the effect of one or more activities over a class or on a longer
term basis.
These assessment mechanisms were developed at the Summer 1996 Workshop
based on experience of the participants. Some examples of these mechanisms
are included in this outline.
Grading of a Cooperative Exercise
This assessment is used in assigning group and individual grades for an
assignment.
Two examples of this type of assessment were given:
- Individual evaluation of a group exercise. This type of evaluation
can both be used to establish individual contributions to a group effort
and obtain feedback on the success of the exercise in terms of group
dynamics. Using individual weights or multipliers, this method of
evaluation can be combined with grading the group exercise to establish
individual grades for each group member. Examples of such evaluation forms
are ones used at WPI and Grand
Valley State University.
- Comparison of individual vs. group work. This exercise involves four
steps:
- Assign an individual homework example.
- Collect homework.
- Have an ad hoc group in-class do the same or similar example and
collect.
- Compare individual work with group work.
Alternately the steps could be done in the order 3-1-2-4 where the group
work is done first and then the individual work.
Formative Assessment During a Course Using Cooperative Learning
This type of assessment is used during a course to elicit feedback from the
students on the outcomes of particular lectures or cooperative exercises.
This feedback can be used to correct problems with the lecture or group
exercise.
Two examples of this type of assessment were given:
- A daily formative assessment could be used for either a group or
non-group class meeting. It involves 2-3 minutes at the end of class
meeting where the students write down answers on a ``2 minute paper''
to a few questions as
feedback to the instructor. The questions could include:
- Did everyone in the group participate?
- Did the group stay on task?
- Did an individual dominate the group?
- What was the most important idea you learned today?
- What is one idea you are still confused about?
- Intermediate or periodic assessment is used at appropriate intervals
during the course, such as after/before an exam, project or out-of-class
activity. When linked to a cooperative exercise this assessment is similar
to mechanisms described for particular exercises. Potential questions are:
- In what ways did you contribute to the group?
- How were your ideas received by your group?
- In what ways did your group work well?
- What problem areas associated with your group need to be addressed?
- To what type of student is the project most appropriate?
- Distribute 100 points over your group members reflecting technical and
group skill contributions.
Summative Assessment of a Course Using Cooperative Learning
The final type of assessment is to summarize the impact of the cooperative
learning activities used in a course. Ideally, the impact is measured
against a similar (control) class in all respects except that the class did
not use these activities. Assessment can either be immediate or longer
term. Immediate assessment is done at or near the end of the course. Such
measures are:
- attendance in the current vs. control class
- retention of students passing current vs. control class
- grades of students in current vs. control class
- performance on common final exams between classes
- student opinions (quantitative and qualitative). Two mechanisms can be
used.
- Standard course evaluations used at a school. While likely not
designed to capture the effect using cooperative learning exercises, their
commonality allows for easy comparison with control class.
- The use of pre- and
post-surveys given at the beginning and end of a class. These surveys can
be used to obtain background information from the students and determine
how student opinion changes over the duration of the class. The
pre-survey and
post-survey used in the CS2005 course at
WPI are one example.
In addition to the immediate impact of cooperative learning activities in a
course, there are the long term effects that can also be measured. Again
the long term effects should be compared for students in courses with and
without cooperative learning activities. Potential measures are:
- grades in future courses
- retention and graduation rates for students
- number of students transferring into or out of major
- increase/decrease in the number of minors
- surveys of student opinion at later points in their career, such as at
graduation time
- delayed exams on course material
- survey of post-graduate employers and schools on student preparation