e-learning ran a regional event for the podcasting for pedagogical purposes SIG on the 11th Nov. I think it was a success, and we’ll be running another event on the 2nd march (using SMS / texting in HE). The following are some of my notes to remember for the next time

• rooms – 6 west was fine, make sure we have booked atleast two break out rooms. Also make sure the projectors are OK in the breakout rooms
• people seemed to find it fine, so same as before, a few signs to help people
• need to have clearer instructions for the wireless network. People didn’t follow onscreen instructions w.r.t re-booting.
• get them onboard before the questions (responseware), so they have a chance to contribute
• get better mics for the presentors … the clip mics didn’t work and lots of sessions didn’t record !!!
• make sure all presentations are available on the presentors machine, i.e., the keynote presentation was not, so could capture using panopto
• e-learning tablet PCs were great, just make sure have software installed and wireless set up.
• better use of the event tag, in particular, pointing it out regularly on slides
• ensure there are poster boards, and encourage the e-leanring team to make a few more posters
• ensure we have some representation from a JISC type service – this worked very well at the podcasting event. If we can’t get a service then perhaps some blurb from JISC Advance or the HEA

This is a question which needs to be addressed as the e-learning team are using some external money to buy a PSP. The answer is relatively straightforward. In terms of background, devices like the PSP are becoming more common place with students (or variants of, i.e., the new generation of nintendo ds). They also offer significant opportunities for the student in terms of lerning. For instance, they have wi-fi web access, they have rss aggregates pre-installed, the can play multimedia, they can capture multimedia, they include chat tools and conference capabilities. In other words, for low level type of uses these are superior to laptops !!

Our purchase fits into a wider ongoing project around the area of using mobile devices in learning and teaching. This will include, Nintendo DS, iPod Touch, Smartphones (blackberry and windows platforms). In particular;

• ensuring Moodle works effectively on this type of device
• activities in face to face teaching can be completed, i.e, use of responseware, sms/twitter, note taking etc.,
• how viable are these as data / evidence collection tools, i.e., capturing info and uploaded to an e-Portfolio

Once the devices are delivered the first activities will be;

• access the wi-fi network and complete a responseware activity (also use a iPod Touch, Nintento DS, Blackberry and HTC).
• complete a number of basic tasks in Moodle (also use an iPod Touch, Nintendo DS, Blackberry and HTC Touch HD). The tasks will include, navigating to a specific course, adding material to a forum, complete a 2 question quiz, opening a web page in the course, reading a word document). We’ll need to make a course up for this test

The outcomes of these exercises will be written up on mobile-learning.blog-city.com, and relevant blogs at the University of Bath.

The following is the raw data (responses) when I asked people at the recent DoS meeting to write down their what questions they’d like answered, or issues addressed in the forthcoming e-assessment review (2010).

These are in no order (typos are all mine) ….

21 responses

————————————

• we might like a system that collects video clips and combine these with other assessments (MCQ, papers) for marking
• use of writing tablets, in exams and part of the marking process
• overcoming the issue of downloading 100+ short assignments from Moodle and providing feedback is a very slow process
• central OMR service?
• e-assessment for mathematics: Maple TA or STACK
• provision of specific central support for departments (not just training workshops, FAQs and helplines) so staff able to work with academics over a sustained period on a one-to-one basis as a new system is introoduced
• are there models of e-learning in which learning and assessment are intregrated, i.e., not having to set aside tutorials for tests only
• tie up and impact with rest of teaching process (i.e., unit assessment redesigned away from high stakes terminal exam during dedicated assessment period and towards smaller value tests given throughout the semester).
• what support (i.e. workshops) will be made available?
• e-feedback
• does the review assume that the university will move towards on-line marking of all coursework?
• e-feedback, for example could a submittted piece of electronic work be annotated (eg using a tablet) electronically and the feedback made available to students
• MCQ for large classes and infrastructure needed
• automated addition of other tools eg plagiarism detection or automated tester for programming assignments
• what about following the OU model of e-TMA’s Quick marking with feedback .
• Integration with TurnitIn
• issues about assessing free text work, eg PhD disserations
• is there a danger of the move to introduce e-assessment tools driving the pedagogy?
• routine and automatic incorporation of plagiarism detection software
• effective ways of providing e-feedback, formatively and summatively, to individuals and groups
• don’t neglect printing costs of printing assessed work submitted online for marking At present met by students !! Parallel paper and e-submission better?
• is moodle the best/most flexible system for organising and running forum discussions for groups of different sizes and abilities
• other technologies (e-portfolios)
• technologies for batch processing of assignments within moodle (batch downloading and storage)
• e-assessment of large MCQs would be extremely useful
• anomymised peer assessment
• MCQ
• student attendance at lectures
• electronic tutorials – students generated unique question, complete online, monitored and assessed
• professional placement portfolios – possibility of submission of reports from external agencies
• anonymous marking / procedures – are they working?
• not sure if audience response system counts as part of e-assessment
• Moodle performance
• automated assessment that goes beyond MCQs, natural language processing, diagram recognition

We provide an automatically generated QR Code on the bottom of our Moodle pages – this allows people to scan the code if they wish to bookmark, or deep link back to the page it they print the page and wish to access via a phone etc.,

This has been available for a number of months, I’ve recieved a few email comments and it will be time to change the wording to reduce the degree of confusion

comments so far …

————-

Furthermore, I often get a bar code square asking me to scan with a
>> cell phone camera.
>>
>> My specific questions are: 1 – what do I need to do to join a forum
>> group. 2 – do I need to do anything with the bar codes?

————-

Many thanks for your speedy reply. I’m logging in remotely from my PC
- and have logged in at moodle.bath.ac.uk – I searched for ‘research
skills online’ but am then asked to scan the qrcode with my phone. I
tried setting up a qrcode, but get no further than this. I’m not
overly technologically minded, so maybe I’m missing something or doing
something wrong?

———————-

The current text on the page is …

Mobile Barcode

Scan this code with your mobile phone camera to access this page online.

To find out more, visit http://blogs.bath.ac.uk/qrcode

————————-

As you can see some of the confusion from the users is they simply don’t know what the QR Code is, and they assume (rightly) based on the text info that they need to scan the code. The initial approach was to minimise the text information. However, given the evidence a new approach would to change the text. ie.,

—————

The url for this page is available in this QR Code

<<image>>

If you scan this QR Code on your mobile phone you can access the page directly on your mobile device. You might use this for quick navigation or bookmarking.

If you are accessing this page on a computer then simply ignore the code.

To find out more about using QR Codes on your mobile device, visit http://www.bath.ac.uk/barcodes

—————————-

The following is are some of my developing ideas based around the initial responses of a survey I’m currently running with our users on the University’s Moodle Service.

The survey runs until 15th November. The aim of this survey is to enable the e-learning Team to gather information on Moodle performance during the start of this academic year (2009-10).

As background, we are aware, although Moodle did not stop working, there were some periods when service performance was very slow.  For instance; during the first 4 days of the new academic year we had the following number of unique users logging on (need to multiple for the number of logins, and issues with concurrent accesses). This is over 40% of our student FTE !!!

More details are available on the Moodle Service Blog (http://blogs.bath.ac.uk/moodle). We also had a number of emails to our e-learning@bath.ac.uk queue. One email in particular was concerned with availability for distance learners. Therefore, I’m keen to identify how the experience was from the perspective of staff and students who were attempting to complete tasks.

The following are the initial respones (still have a week until the survey closes). We’ve over 150 responses The methodology was to collaboratively develop the questionnaire (big thanks to the e-learning Team), promote this via a post on the Moodle Service Blog, a resource in Moodle (the post log-in page), a forum post in the Staff Area course, and an email to the University of Bath’s Directors of Studies (UG, PG). I’ve not incuded any of the qualitative responses.

It is evident the large number of responses are from students (91 out of 154).

The intention was to capture how satisfied people where with the Moodle Service over the key period. The following ar ethe responses when asked overall how satisfied were you with the Moodle Service. 

It was very pleasing the majority of respondents (59%) were either satisfied or very satisfied with the Moodle Service. Even more pleasing was only 4% were not satisfied at all. It can be interpreted the vast majority of the users were satisfied with the service.

This trend continued with specific questions concerning responsiveness and availabilty. Overall, for both these indicators evidenced that the “very satisfied” category increased. In terms of overall availability, there was a higher number of people very satisfied with the service (32%).

An interesting observation is pattern for satisfaction with respect to responsiveness shifts slightly towards the satisfied / neutral choices. This isn’t a surprise as the system was available over the period, i.e., it didn’t stop working whereby it was not available for all users. However, at some periods it became very slow, i.e., the responsivess declined. This is captured in the responses.

Recommenations

The findings from the survey suggest we (the Moodle Service supplier) still have a significant way to go interms of the quality of our service. Yes, there are a significant number of users who are very satisfied with the service, however, we have a small number who are not at all satisfied. This might be a small number, however, they can not be ignored. The target for the start of the next academic year will be ensure the service is more responsive so we have an improvement in the responses.

There will be a number of new starters in the team in the new year. These will all have a set of induction tasks to complete as part of their probation. Apart from the maternity cover, who I assume will have a slightly different probation period (need to check !!).

In terms of the ESSD posts, the aim of the induction tasks are as follows;

1. let them get upto speed with our install of Moodle 1.9.6, custom code, and the specific development of the Moodle-SAMIS integration

2. get them familiar with the process by which we run our Moodle install, i.e., development box through to live, including subversion control, and testing procedures

3. make them familiar with the relationship between the different parts of the team; ESSD, LT (central & dept), and the use of RT and Trac

4. expose them to working collaboratively within the ESSDs, other members of the Team, BUCS and the wider UoB community

5. objectively identify they have the required skills, knowledge and approach as outlined in the job description to be able to play an active and appropriate role within the team. In other words, evidence that they have successfully passed probation, and indicate from an early stage their progress through the probation period.

Therefore, the induction tasks should be;

1. very well defined
2. achievable within the time frame
3. ring fenced time
4. mapped back to the required skills / knowledge in the JD
5. can be monitored and this monitoring is transparent and objective
6. include set milestones

Ideas for tasks, and timeframes;

• 15 minute presentation on how the SAMIS-Moodle Integration (Block) works, and suggestions for feature enhancements from the user perspective. This is a four week piece of work
• Moodle housekeeping – removal of old users, and hiving off of log tables. This is a two week piece of work
• 15 minute presentation on risk assessment on current system architexture (hardware & software set up) in terms of trends in user growth and expectations for start of 2010-11 academic year. Three week piece of work

>>> mid probation >>>

We are continuing to use RT to log all queries to e-learning. This has even involved a member of the team adding telephone queries. This is a really nice addition

A question often asked by me as the Head of the team is what’s happening in terms of the number of queries entering the process. Is it stable, increasing or decreasing? This will give an indication of pressure and requirements in terms of resource allocation.

The following displays the number of tickets created per day since 1st Sept 09 on our system.

It shows the help desk get a large number of queries (over 400 in the last two months), it also shows the peak period is October (as you’d expect with the start of the new academic year), and the underlying trend is starting to fit the september pattern of a few emails a day.

This is good news in the sense it looks like we’ve got through a peak period (thanks Team), so we can re-think the required resource level of front desk staff.

Please take this with a pinch of salt, i.e., it gives us the trend data. However, we’d need to map this against resolved in the same period, time taken to resolve and type of ticket to start to unpack a clearer picture. Much of this I simply haven’t the time to answer. Therefore, for now, I’ll leave it that RT queries (volume) will start to fall back to a much lower level, indicating we can release people to start to work on other projects.

Met up with Jules to discuss e-assessment (objective) work which I need doing.

Have agreed he’ll write a 3000 word discussion paper (for publication on OPuS), around the following questions;

1. what is e-assessment (objective testing)? (about 600 words)

2. how might I use it in my teaching and learning? This would include scenarios, evidence from UK HE and publications. The aim is for the audience to be won over to the pedagogical benefits of objective testing for formative purposes. They can reflect on how they might use it in their teaching.

3. objective testing in Moodle 1.9? (a few hundred words). This is the authors perspective of the use of other tools in Moodle (not just quiz) for objective testing, i.e., the lesson.

This will be completed by Dec 09. Assuming agree on the cost (;-)).

The rationale for this is, in terms of support e-assessment (formative, objective testing) at Bath, we have the FAQ which contains lots of material on the Moodle Quiz (http://moodle.bath.ac.uk/faq/category/1/quiz.html), and the e-book available through the staff area. We also have links to guides on question design (http://esu.bris.ac.uk/esu/e-assessment/writing_e-assessments/index.htm) and material on planning your online assessment (http://esu.bris.ac.uk/esu/e-assessment/planning_online-assessment.pdf). Finally we have policy statements on using Moodle for formative assessment only (http://blogs.bath.ac.uk/moodle/moodle-policy/#high-stake). However, we don’t have why and how to effectively use e-assessment in your teaching. So it is the last piece of the puzzle.

We discussed the idea of workshops etc., agreed we’d need to look at this for semester 2.

Finally, discussed a role in the upcoming e-assessment review. This will involve a data collection activity (who is using e-assessment, why, and what are their expereinces). So it would be nice to use an external for the semi-structured interviews.

Also, given the likely outcome is, we need systems to provide e-assessment on (in terms of robust, scaleable solutions), I’d like to find out if Moodle 2.0 is future proof interms of quiz, and online submission functionality. Jules could be commissioned for this report, i.e., is it fit for purpose from the academic / admin perspective. Then I’ll get a technical review from the e-learning / BUCS team. A potential model is to provide a second install of Moodle for high stakes summative (objective) testing and online electronic submissions.  So ensuring this type of activity can be provided away from the general use of Moodle. Plus, the institution could own this, so it could be on a separate upgrade path to Moodle, and include our own adds up.

Just been to a Mech Eng Sypodium software demo in 4E 3.38.

Firstly, organised by an academic (Nigel XXXX). Who had been using it and thought that others might be interested in using it as well. Especially, given 4E 3.10 has the same set up. In all 10 people attended, not including myeslf and Cher (AV) who’d been invited.

Session was very good, lots of people tried it, lots of interest. We finished up mulling over different scenarios of use, when/where you might use visualisers etc.,

A point to include is on the timetable software can it would be useful to have a link to uses / support pages. So if you know you are going to use the room you can check out how to set up the equipment.

Also, I learnt the following;

1. visualisers can be recorded — need to find out how
2. panopto to broadcast live (i.e., to other rooms) — need to meet up with Sacha and find out more.

Introduction

The aim of this report is to inform to the Head of e-Learning, the e-Learning Team, the LTEO and the wider community at the University of Bath if the e-learning staff development programme (central provision of events) is achieving its broad aims, and feed into planning decisions for the next academic year.

The need for an appropriate and effective e-learning staff development programme is highlighted by an objective in the new Learning and Teaching Strategy . The strategy states the following objective is owned by e-Learning; “develop appropriate and effective e-Learning development and support programmes for staff and students”. Further more, the aims of the e-Learning Team at the University of Bath are;

• Helping staff to integrate e-Learning into their programmes
• Sharing innovation and good practice
• Developing an e-learning community
• Promoting and evaluating e-Learning
• Developing e-Learning tools

To help achieve these aims we run a number of central staff development events (workshops, seminars, blended courses and coffee breaks) throughout the academic year (see: http://www.bath.ac.uk/learningandteaching/events-workshops/?dept=elearning)

In terms of our broad aims we need to identify if  “our staff development programme achieves the broad aim of developing staff capacity in e-learning? does it share innovation and good practice while encourage the develop an e-learning community? This report attempts to answer these questions for the 2008-09 period.

Background

The following are the descriptive statistics for the e-learning staff development programme (2008-09). It is evident we have run a large number of events over the period (28) and cancelled 20 sessions. In total, 148 particpiants attended our centrally provided events.

Table 1: Events (by type)

Semester 1 (ran/canceled)

• workshop 10 / 7
• seminar 4 / 2
• blended course 1
• coffee breaks 0 / 0
• elpf 0 / 0

Semester 2 (ran/canceled)

• workshop 2/6
• seminar 6/2
• blended course 2/1
• coffee breaks 4/0
• elpf 0/1

Table 2: Events by broad focus (all sessions)

Semester 1 (moodle / non moodle)

• Workshop 12 / 5
• Seminar 0 / 6
• Blended Course 0 / 1
• Coffee Break 0 / 0
• elpf 0 / 0

Semester 2 (moodle / non moodle)

• workshop 5 / 3
• seminar 0 / 8
• blended course 0 / 2
• coffee break 0 / 4
• elpf 0 / 1

Table 3: Total attendance (average per session which ran)

Semester 1

• workshop 53 (5.3)
• seminar 14 (3.5)
• blended course 17 (17)
• Coffee break n/a
• elpf n/a

Semester 2

• workshop 7 (3.5)
• seminar 41 (6.8)
• blended course 7 (3.5)
• coffee break 9 (2.25)
• elpf n/a

Analysis

The first question is, does our staff development programme build capacity in the individuals who attend our sessions and have they applied it within their teaching?

A survey was run (30th July, 2009 to 5th September, 2009) for staff who attended one of our events in the Semester 1 period (Sept 08 to Jan 09). The time delay was to give participants a number of months after the event to apply the knowledge and skills gained from the e-learning event before evaluating impact.

The respondents were strongly bias towards those attending the “Introduction to Moodle” workshop (13 out of 16). This workshop is designed for those new to Moodle, to act as a heads up, start uploading content, and managing their courses.

The survey included questions to inform us about longer term impact. For instance, to what extent do you agree with the statement, “I have applied the knowledge and skills which I developed during the workshop in my teaching and/or work? The responses indicated 6 strongly agreed with the statement, 5 agreed, 2 were neutral and 1 disagreed. Overall, it appears the majority would argue they have applied the knowledge. When asked to explain why, the extracts indicate a positive impact and implied effective embedding within practice.

“I have currently only used the basic applications in Moodle but plan to use more in the future”

“I have never used Moodle before, and the introduction sessions makes my life a lot easier with moodle”

“I have set up a moodle site”

“I was introduced to Moodle during the workshop, and at the time, I was very early on in my post. But when I came to use Moolde for my job, I remembered most of the training and my project could not have taken off without the use of Moodle”

In terms of exploring peoples views of the quality and worth of the session I asked two questions. Firstly, would they recommend an e-learning workshop to a colleague? Secondly, has attending the workshop motivated you to attend more events by the e-Learning team? The first question is based around the concept that a person is only going to recommend a course to a colleague if the experience was good as their reputation is attached to the recommendation. So if you did not enjoyed the experience, or questioned the value added of attending in your teaching and learning you are more likely not to recommend attending. The responses to these two questions were, 8 respondents would (57%) would recommend it, while 6 would not. In terms of motivating them to attend another session 9 people it did, however, 5 it did not.

A further question tried to understand why people responded as they did. I was pleased to observe there was no clear message emerging in terms of participants perceptions of low quality or little value added from attending. In fact, there was no comments around this issue. There were a few points about finding time to attend other sessions or there not being any appropriate to their need. One comment needs actioning within our designs for the blended courses. The view is not new, and it does support the views of many of the session leaders. This is the issue of motivation, if people aren’t motivated to participate on blended courses the learning activity becomes meaningless for the others.

The final question focussed on identifying if people engaged in our support materials after attending the event. The assumption being if they followed up the session by accessing more resources and assistance this would indicate an increase in staff capacity, and sharing good practice. The responses suggested a number of people (at least 9) engaged with the following resources over the last three months;

• accessed the e-Learning pages on the LTEO web site: 7
• visited ine of our supporting (blog) web sites, such as Moodle Blog, ARS, SMS or QR Code: 2
• emailed e-learning@bath for assistance: 4
• read an e-learning case study from the LTEO web site: 0
• visited the staff area in Moodle: 9

A slight concern is the lack of awareness of our case studies. This is a concern as we intend these to be one of the primary mechanisms for sharing good practice.

The questions concerning does our programme share good practice at the University, and does it help develop the wider e-learning community can be addressed in a number of ways. In particular, for this to be effective we need to explicitly sign post this to participants. A simple method would be to;

• include case studies from practitioners at the University of Bath within the sessions. These should be provided as linked examples from the case study web site (http://www.bath.ac.uk/learningandteaching/themes/e-learning/casestudies).
• the session should require (if hands on) the participants accessing the social (community) space. They should update their profile, be directed to discussion and resource areas.
• at the end of the session, any outcomes should be made available through the social (community) space. Participants should be made explicitly aware of the resources.
• at the end of all sessions there should be the follow up email, thanking them for attendance and encouraging them to visit the social (community) space with specific questions raised from the session.

A review of a number of slides, and discussions with individuals within the team indicated the above approach is not commonly deployed across our courses. An action for the Head of e-Learning is to ensure it is. A caveat is we still haven’t decided on the social (community) space. There have been discussions around using Moodle or a social network tool (Crowdvine, Ning). An action is for us to agree, asap.

Recommendations & where next

The message from the descriptive statistics appears to indicate a large number of sessions were canceled (1 in 3), which were mainly the workshops. This will be for both supply and demand reasons.  When these are canceled the session leader follows up with 1-2-1 sessions. In terms of the next session this implies we should reduce the number of duplicate sessions we make available, and shift some workshop topics into seminars.

The descriptive statistics imply Moodle development sessions tend to be in the workshop format and are run mainly in Semester 1. While non-Moodle topics tend to focus on shorter formats and run in Semester 2. A recommendation would be to even out the balance across the academic year and by type.

We need to put more effort in weaving in our case studies and support spaces if we are going to use our staff development programme as a vehicle for meeting the sharing practice and community building aims of the e-learning team.

In terms of the methodology, the survey needs developing a more effective way to measure the impact on sharing practice, and community building. Also, we need to include more varied data collection techniques (selected interviews, focus groups) to triangulate these findings.

Timetable

• Data collection during November 2009 for those who attended sessions in Semester 2 (Feb 09-June 09)
• Data collection during June and July 2010 for those who attended sessions in Semester 1 (Sept 09-Jan 10)
• Next Annual Report: Due September, 2010