Evaluation should not be an afterthought but a feature of your course planning, embedded within the overall design of the course. It may even involve the active participation of students (e.g. by asking students to complete learning logs or reflective diaries over the duration of the course). If you are planning to involve students in evaluation activities, they will need to be informed at the beginning of the course about their role and contribution to the evaluation effort.
Defining the purpose
The first step is to define the purpose for the evaluation:
- Diagnostic (pre-course): is the purpose to learn more about our students and their reception to blended study methods? This may touch on their prior experience in using learning technologies to support study activities.
- Formative (during the course): to enable us to reflect on the impact of the online learning activities during the delivery of the course – helping us to make adjustments to instructional methods influencing both online and class-based activities?
- Summative (post course): to help us to review the impact of the course design and delivery methods on student learning, with a view to transferring lessons learned to future courses?
Focusing the evaluation
We then need to define our focus for the evaluation and key questions. What aspect of the blended experience do we want to explore? For example:
- Appropriateness of the technology in supporting study activity (targeted learning) – outcomes, engagement levels & interaction patterns.
- Suitability of the online content resources & activities in supporting students and promoting active learning.
- Levels of student engagement in online activity, relating to discourse and interaction patterns.
- Clarity of instructions and levels of support (administrative, technical, pedagogic) to help students tackle the targeted learning activities.
We may also wish to review the interrelationship between online and face-to-face elements of the course i.e.:
- The degree to which the online and class-based study methods complement each other – reception of the study methods & degree to which they are viewed by students as interrelated.
- The pedagogical effectiveness of the course design in helping learners to reach the targeted learning outcomes.
As a general rule, the evaluation focus should be aligned with the targeted learning outcomes for the course – what you set out to achieve with the initial design of the course – and consequently linked to the overarching course objectives.
The case study in the video below is an example of peer-evaluation. The evaluation was negotiated to look at specific aspects of online teaching, with an open discussion to support professional development in those areas.