The University of York has completed its retender process for an institutional e-learning platform, awarding a four-year contract to Blackboard International BV, which will commence in January 2016. Blackboard will therefore continue to work with us as an educational partner and supplier of e-learning services.
The retender decision was approved by University Teaching Committee in December 2013 and formally signed off by our Supplies Office on 17th January 2014. This marks the culmination of a 6-month retender process, which commenced at the beginning of July with the publication of a contract notice in the Open Journal of the European Union. Preparations for the retender pre-dated that publication notice of course, with an e-learning working group from University Teaching Committee, constituted by our Pro-Vice-Chancellor for Teaching and Learning Professor John Robinson in January 2013 to lead a consultation process on our future requirements for e-learning.
This blog post summarises the key events leading up to the contract award, and will hopefully shed some light on how the retender process was conducted and the final decision was reached.
(i) Defining our requirements (January – June 2013)
The consultation process involved a series of open forum meetings with staff and students through groups such as the Learning and Teaching Forum, Distance Learning Forum, Departmental VLE Coordinators and Learning Technologists, with participants invited to feed back their views on a list of e-learning issues in the following consultation document [doc]. The combined feedback from staff and students confirmed:
- the progress that the University has made in embedding the use of learning technologies in course design and delivery;
- a continuing need for a centrally-supported online learning environment to be available to underpin this practice. (N.B. This had not been assumed from the outset, and the consultation process purposely invited feedback on a range of technology scenarios that we might pursue in the future, from a monolithic learning management system to a more flexible (disaggregated) architecture based on the provision of a series of e-learning tools.)
Feedback from the consultation process helped to inform the updating of our specification of requirements for an online learning system from the original tender specification [doc] of May 2004, and the discussions also contributed to the establishment of a fresh institutional vision for e-learning, with a statement on our e-learning pathway approved by University Teaching Committee in June 2013. The statement summarises our ambitions for the use of learning technologies in teaching and learning, focusing on our commitment to standardising the provision of rich learning resources for our students across all study programmes, whilst also using technologies to support active learning opportunities for our students, based on participant-controlled online learning activities. The full vision statement is available here: http://tinyurl.com/elearning-vision-statement [pdf].
(ii) Framework for the retender process
The VLE retender process was conducted according to a ‘restricted procurement’ process led by the University’s Supplies Office, with University Teaching Committee’s e-learning working group retaining ultimate ownership and oversight of the process. The restricted procurement process was adopted due to the large number of expressions of interest that we anticipated receiving from providers, and was intended to help us to narrow the field to identify our preferred supplier. Following this process, potential suppliers are selected using a pre-qualification questionnaire, in response to a contract notice published in the Open Journal of the European Union (OJEU) inviting expressions of interest. A minimum of five short-listed organisations are then selected to submit a full tender, and after evaluation of their responses and a further round of short-listing, preferred suppliers are invited to deliver on-campus demonstrations of their solutions. From the evidence presented through the written responses and on-campus demonstrations, the winning tender is then selected.
(iii) How the retender process worked out (July – December 2013)
A contract notice for an online learning system (an intentionally open description of what we were looking for, rather than a narrowly defined virtual learning environment, so as to encourage as wide a field of suppliers as possible) was published in OJEU at the beginning of July 2013. This led to 39 companies registering an interest in the retender, with 14 pre-qualifying questionnaire (PQQ) submissions subsequently received. The PQQs focused on the financial health, resourcing and expertise of the companies in relation to the contract theme and were reviewed by an evaluation panel comprised of representatives from the Supplies Office, E-Learning Development Team and Information Directorate. Of the 14 submissions received, six were deemed to be promising enough to be progressed through to the next ‘invitation to tender’ (ITT) stage, involving a full tender response to the revised specification of requirements document [docx]. The shortlisted companies reflected a range of expertise including e-learning specialist providers, suppliers with hosting experience for learning management systems (LMS) and software houses with LMS development experience. Both commercial and open source solutions were offered by suppliers and considered by the evaluation panel at this stage, and we also considered self-managed and hosted options for the online learning system.
The evaluation of the ITT responses focused on the extent to which the proposed solutions satisfied the mandatory requirements for the VLE (pedagogical and system functionality) and met the ‘highly desirable’ and ‘desirable’ requirements set out in the advertised specification document. The evaluation also focused on proposed costs for the solution, and service requirements including migration and support services from our current VLE system (Blackboard v9.1 SP12) to the new environment. It is important to note here that the migration effort was only a small component of the costs that we were looking at, which also addressed development estimates (where desired functionality was not already available) and on-going service management costs too. This was intended to reflect holistic service management costs for the full term of the four-year contract (not just set-up charges in the first year) and in this way offer a level playing field for all suppliers bidding for the contract, rather than unduly favouring the supplier of the current system in use at the university.
Taking all these factors into consideration, the scoring revealed a very clear picture. Blackboard’s proposal was deemed to be compliant with all key requirements and scored the highest in terms of functionality and was also costed within the advertised budget.
The Blackboard account team were subsequently invited to the next stage of the retender, to address an open audience of staff and students on their solution and to address a challenging scenario on content management, followed by more detailed questioning by the evaluation panel.
Overall, the Blackboard solution was well received, as reflected in the feedback submissions from the staff and students who attended the demonstration, although some criticisms were made over the stability of Blackboard’s XpLore functionality (a cross-platform learning object repository and authoring environment in beta testing / development at the time of the demonstration) and the degree to which the suppliers met the precise requirements of the test scenario. Notwithstanding these criticisms, no issues were raised by the audience that would prevent the award of this contract to Blackboard.
(iv) Additional accessibility testing (November – December 2013)
As an additional action and not formally a part of the tender evaluation process, the Blackboard platform (v9.1. SP12) was evaluated on accessibility and usability dimensions by Computer Science’s HCI group using the PROTEA (Protocol for Testing e-Accessibility) methodology, employing a set of test scripts developed by the E-Learning Development Team. The preliminary results of the accessibility tests that were run highlighted issues relating to the accessibility of Blackboard’s content editor for blind and partially blind users employing screen reading software such as JAWS (v. 15.0). However, through dialogue with Blackboard’s web accessibility team these issues have now been resolved and we have confidence that Blackboard’s internal quality assurance practices for accessibility testing of maintenance work (patches) and new product releases are compliant with UK standards and legislation, as outlined in the Equality Act of 2010.
What happens next?
It is important to note that the retender outcome does not mean a ‘business as usual’ / no change agenda with regard to the technology that we use and the way that it is deployed to support learning and teaching activities. Our vision statement on the future of e-learning at the University invites us to consider ways in which we can establish an elegant repository of rich learning resources across all taught programmes and to promote pedagogic change, inspiring our teaching staff to design in to their courses ‘participant-controlled’ learning activities which offer active learning opportunities to our students. There are implications here for the way that we design and share our teaching materials, and employ technologies in the design of independent and collaborative learning activities for our students. We will therefore need to invest further time and effort in developing the technical infrastructure of our institutional learning environment, whilst helping staff to make the best use of the supported tool-set to achieve these goals. Blackboard will be a partner in this endeavour.
In closing, we should also note the arrival of a new Vice Chancellor, Professor Koen Lamberts, at the University. Professor Lamberts has initiated a process for developing a new University Strategy. The opportunities presented by e-learning and our institutional technology investments will be an important part of the Learning and Teaching strand of strategy development, governing learning and teaching developments moving forward.
E-Learning Development Team Manager