Day 2 of the bett UK 2015 conference Technology in Higher Education track (PDF agenda here) focused on innovative uses of technology to improve the delivery of teaching and learning and provided an opportunity to showcase the work that we have been doing at York to support the development of electronic assignment management and marking processes. In a presentation entitled ‘Enhancing the assessment experience through closer integration between the SRS and VLE’, I presented on the context and rationale for developing our own customised e-assignment workflow, drawing on the integration work that we have undertaking between our student records systems (SITS Tribal) and our Blackboard virtual learning environment. The presentation slides are available here.
Our students in their feedback to last summer’s Technology-enhanced learning survey clearly supported the case for electronic submission to be introduced for all assessed work at York, but there has been long-standing frustration amongst teaching and administrative staff regarding the maturity of tools available to them to manage the allocation of work online and the marking of student work. We are not alone in this respect, and the Jisc in its recent Electronic Management of Assessment (EMA) landscape review report has highlighted the ‘relatively limited nature of the core product set’ for institutional e-assignment solutions and the poor level of integration which exists between assessment systems (Jisc, 2014: p.12). This has resulted in the Jisc setting up its own consultation exercise with the HE sector to explore how technology can support the full assessment lifecycle, encompassing electronic submission and the marking of assignment and the delivery of feedback to students. It is too early at this stage to comment on the outputs that might come out of this consultation process, which has been focusing initially on the identification of the key challenges to EMA, rather than the development of an open source e-assignment system, which Jisc has previously attempted to do.
We may yet see progress though on technical development for EMA, with commercial vendors looking to develop their assessment systems to common standards to help support better integration and data flows across enterprise systems. We are particularly encouraged by the Blackboard / SITS agreement to develop to the LIS 2.0 standard – announced by Blackboard at the 2014 Blackboard Teaching & Learning Conference – which offers the prospect of a common pathway for marks and feedback to be transferred across these systems and support towards a full assessment cycle. If this delivers all that it promises, it may provide a pathway for summative grades and marks to be pushed from Blackboard’s Grade Centre to SITS eVision, and would enable us to make greater use of Blackboard’s anonymous marking tools, by using exam IDs stored in eVision rather than the randomised IDs that Blackboard’s current anonymous marking tools employ. Blackboard are set to announce details of the development work they have been doing very shortly – so we shall see!
Phase 1: Delivering enhancements to the administration and management of assignments, through the generation of individual sheets containing Exam ID and disability sticker information, and a batch upload facility for feedback and marks to be pushed into SITS.
This development phase was completed last year, with the departments of History, Law, Politics and the Management School employing this functionality in a series of carefully controlled pilots. The pilots were successful, with a number of benefits being reported by departmental administrators including faster allocation of assignments and feedback sheets to markers and a smoother return of marks and feedback to SITS eVision. A summary of this phase of the project and the key outcomes from the pilot projects is available here.
Phase 2: Delivering improvements to the anonymous submission and feedback workflow. In response to the feedback received from the summer 2014 pilots and a wider consultation exercise conducted with departments, additional development work was undertaken over the summer to support enhancements to the student experience of e-assignment submission, as well as the academic experience in managing feedback and marking processes. Enhancements include a student view of all previously submitted files and a facility for markers to annotate / insert comments into file submissions as student feedback, which can then be batch uploaded to SITS. These new features have been rolled into the e-assignment workflow, and this is used by a wider set of departments over the spring and summer terms of this academic year (2014-15). We will be evaluating departmental experiences with this enhanced tool-set and reporting on this in due course.
Phase 3: Enhancing the management and practice of online marking. The aim for the third development phase is to address the marker and moderator experiences. We will be looking to develop processes to allocate automatically feedback sheets and assignment files to markers, and to generate plagiarism checks as well. We are also keen to look at reporting tools and the development of a moderator’s view of the marking process. In this phase it’s important that we avoid ‘reinventing wheels’ if possible, and we will be looking at ways in which we can make greater use of the Blackboard anonymous marking tools. The challenge will be to find ways of mapping exam IDs stored in SITS to Blackboard’s anonymous marking workflow.
Finally, in a related project, we are looking at ways in which we can present a student’s marks and feedback for all assessed work (formative and assessment) in one place, to support a reflective process across a study programme and the feed-forward of lessons learned from previous assessments to new ones. There has already been some excellent work conducted across the sector in structuring reflection on feedback (e.g. Westminster’s e-Reflect project) and the Jisc has published some useful guides – and we would like to draw on this practice in developing our own institutional provision for student reflection on the marks and feedback that they receive across their programme.