Planning for evaluation
With a clear view of the purpose of the evaluation and the key research questions and evaluation methods or tools that you will employ, the next step is to proceed with the development of an evaluation plan. Developing a plan will help you to consider how the evaluation methods will be applied across the life cycle of the course, identifying key actions and the stakeholders who will be involved, as well as the timing for the activities that will take place. It will help to draw together the threads of your thinking into an action plan.
Developing a plan
In developing your plan you will need to note down:
- The purpose and focus of evaluation: aims and focus of the evaluation, which should link back to the overall course objectives and ideally be mapped against them. The rationale for the evaluation should be clearly stated, as this may need to be communicated to stakeholders.
- Key questions: derived from the evaluation themes that you are seeking to explore (e.g. your focus may be on activity design – with evaluation questions directed to the sequencing of the online tasks within the course and their relationship to the class-based learning over the duration of the course).
- Stakeholders: identifying the groups of people that will provide feedback (e.g. students; tutorial team; administrators) in support of the evaluation, and the people who will manage the evaluation process (e.g. instructor; tutorial team; independent researcher; VLE team) and lead on the evaluation tasks and be responsible for collecting data. Do not underestimate the challenges of getting students to engage in the evaluation process. Survey fatigue is a common problem that often affects response rates, unless requests for feedback are coordinated with other courses and followed up in the right way. Student workload (assessment deadlines) and evaluation requests from other courses should therefore be weighed up, when considering the timing of your own evaluation activities. Ethical issues in engaging students in evaluation activities may need to be considered too.
- Time scales & dependencies: detailing when the evaluation activities will take place (e.g. diagnostic evaluation taking place before the start of the course; formative evaluation during the course with reflection on action; summative evaluation at the end of the course) and the key dependencies – i.e. what needs to take place to ensure the effective completion of the evaluation tasks (e.g. preparation and pre-testing of evaluation instruments; establishing criteria for sampling of cohort for focus groups etc.; availability of key stakeholders to manage & perform evaluation tasks; determining when to communication of evaluation tasks to ensure students engagement etc.; resourcing for capture, typing up & analysis of transcripts for focus groups; arrangements for capturing course stats & activity / participation logs). Zero measurements or baseline collection of data will of course need to be undertaken before the delivery of your blended course.
- Instruments & methods: outlining the evaluation methods that you will use and the data that will be produced, which should link back to the key research questions that you are seeking to address. You should also clarify here your approach to analysing the data – particularly when you are dealing with multiple sources of evidence.
There’s no prescriptive method for how you draw up your plan, but you may find it useful to document your thinking. A template is provided here:
- Evaluation planning template [.docx]