Posted on behalf of Richard Walker
The presentation reported on evaluation data from an MA Research Methods module at the University of York, which employed a combination of short-answer (essay) and multiple-choice style questions to assess student learning, introducing an online examination as an alternative to the traditional paper-based assessment. The examination was taken by 165 international students simultaneously across three different PC classrooms, using a combination of thin client and VLE technologies to lock down browsers and control web access to the examination only.
The switch to an online assessment mode was partly driven by the need to assess a large cohort and return marks in a short period of time while the instructors were busy teaching other modules. The final assessment took the form of short answer (multiple-choice) and longer open-ended essay style questions. Students completed a mid-term mock exam online as preparation for the high stakes exam and also were given access to dummy questions and a video on the exam format, which were made available on their VLE programme site.
Students’ reception of the online assessment experience was evaluated through a mixed-mode approach, based on a survey and focus group interview. The survey instrument incorporated question items from a previous study led by John Dermo at Bradford University (2009) and sought to draw comparisons between the cohorts. The results reveal that computer literacy is not a great predictor of acceptance of online assessment methods, but prior exposure to e-assessment methods does appear to have some effect, and this preliminary study highlights the need for greater socialisation of students to this new assessment method. Students confirmed the benefit of e-asssesment in delivering greater consistency to marking practices, but highlighted a range of practical considerations from the length of the exam to the noise of keyboard typing and proximity of work stations in the PC classroom, which require greater attention in future e-assessment examinations. The instructors will seek to evaluate future cohorts and report back on the data to see how tweaks to the assessment design and preparatory arrangements impact on levels of student acceptance for this assessment method.