CRES Tutorials
Multiple choice questions have a number of perceived limitations. They are difficult to write and are not generally satisfying to either teacher or student when testing any but the most routine knowledge. When used in a Keller Plan course we found that students tended to cycle throught the exams quickly. In our Keller Plan courses, where the tests are administered by computer, we called the result the "arcade effect", for it seemed that many students reached a stage where the tests were treated more like an arcade game than as a serious academic exercise.
It was the "arcade effect" in the Keller Plan courses which orginally led us to consider the use of more traditional problem type questions. The technical problem, of course, is how to handle the free form response from students. The solution that we have adopted is so simple as to be embarrassing: after the student answer has been "submitted" the computer asks the student a number of simple yes/no questions about the answer. The practical effect of these "critical review" questions is that the student marks their own answer. The "critical review" questions may be arranged in a tree structure so that a variety of possible student answers will result in a pass, thus facilitating the use of questions which have no "right" answer.
As mentioned above, CRES (Critical Review Examination System) was originally conceived as an assessment device to be used in conjunction with a Keller Plan course. It was the students who first alerted us to its tutorial potential when we learned that a preferred strategy was to use the examining system as a tutorial early in the study of a module. Conversations with students conviced us that using CRES purely as a tutorial system was both feasible and desirable. We were also encouraged in this approach by the literature on "mastery learning" which suggested that short, criterion referenced tests could be used in conjunction with other teaching methods to provide substantial improvements in student performance.
The first use of CRES as a pure tutorial system was in International Law at the University of Sydney in 1992. Students were given a choice of using the CRES tutorial system or attending ordinary tutorials. Almost exacly half of the students chose computer tutorials. The 'computer' students did marginally but consistently better in two examinations.
The author has received a National Teaching Development Grant from the Committee for the Advancement of University Teaching to construct CRES tutorials in additional subjects. Current projects are contract, company, evidence, legal research, real property and intellectual property.