Part of our e-assessment review is to investigate current practice of Moodle for objective testing at the University – in particular, high stake, summative objective tests, delivered via invigilated PC rooms. We are aware of lots of use of Moodle quiz for formative, and low stake summative (mainly undertaken in non invigilated spaces). This is great, and we’ll write a landscape / snapshot paper on people’s use of Moodle Quiz from this type of activity.
In terms of our set up, performance is fine on the service, and it should scale in the forseeable future. In terms of demand for high stake, summative, objective testing this is constrained by the fact there are very few rooms available on campus for this invigilated activity.
So a recommendation might be to change our policy statement on around high stakes summative objective assessment activities to “you can do it, we aren’t responsible, this is the support stuff ….”
However, it is really useful to identify what are the issues of other institutions who have used Moodle for summative objective testing.
Survey to UK HE Moodle List
Of which 5 had used Moodle to support summative, objective testing. A few of these got to very large numbers sitting the test concurrently (200 students). This is significantly larger than capacity at Bath (about 100 across a large number of GTA computer labs).
Out of this 5, 3 had encountered problems when running the summative tests. However, the listed problems focussed not on hardware / software being unable to support this number of people, i.e. database problems etc., But more on user awareness, for instance, Quiz settings, quiz design, managing the process (staff perspective) with human error leading to gradebook deletions etc., The solution in all cases was clearer guidlelines and advice to staff when setting up the summative objective test.
Interestingly, based on potential confusion, lack of demand and more work people had not set up a second install of Moodle dedicated to summative exams. Although many have looked at server infrastruture with load balancing, clustering etc., However, it could be argued this should be good practice given the likely need from general use, online submissions etc.,
Change policy from can’t to can, but you (academic) is responsible
put in the approrpriate online guideance and recommendations (based on those available from Bristol)
set clear expectations of the role of e-learning with this process, we do not invigilate, we do not enter the questions, we just take people through the process, make them aware of risks and limit these risks. We do work with academics to continue to identify need for this group, and work to ensure needs are met within Moodle.
set up a high stake summative objective test register … so people can tell us when they are doing them 🙂
Given the interest in objective testing, the limitations of space, I’d recommend further investigation into optical mark reading technology