What is a proxy for student engagement? It appears access data to learning resources doesn’t cut the mustard

I was traveling to an onsite visit so I had a great opportunity to catch up on some reading. Top of my Learning Analytics reading list was Colthorpe, Zimlardi, Ainscough and Andersen (2015) – “Know thy student! Combining learning analytics and critical reflections to increase understanding of students’ self regulated learning in an authentic setting

I was particularly interested in this article as I’m starting to explore what are appropriate data triggers for interventions?

The key message I’ve taken from the article is the potential to include non-discipline questions with low stake summative assessment, and the early submission of these meta-learning assessments are very good indicator of academic achievement. While if we consider the strategies of the self-regulated learner, our use of access logs to learning resources can be a misleading trigger for an intervention.

The intention of this post is to partially unpack the above.

Colthorpe et al., (2015) applied Zimmerman’s three stage self-regulated learner model (forethought, performance, reflection). Their analysis identified student access to learning resources (in this case the lecturer recording) was not a good indicator of academic achievement, and would be a poor intervention trigger.

The exploration being a high performing self regulated learner has set their goals and motivations in the forethought stage. Consequently, if they access the lecture recording and they do not achieve their desired learning outcome, they’ll shift to alternative sources, ie., textbooks. This can be contrasted by an individual who has a low self-regulation measured by them not reflecting on the effectiveness of their previous learning strategies, and consequently, have not developed a range of effective approaches. In this scenario, they will simply continue to access the lecture recording as they have no previous experience to draw upon to seek alternative sources. The above illustrates two extremes, but suggest access clicks on resources may not be a good indicator of a struggling student. For instance, even an effective self regulated learner may return to re-watch sections of the lecture recording.

The article evidences a good indicator of academic engagement is the early submission of meta-learning assessment tasks. Where these tasks are designed around the timetable of discipline specific assignments. The structure of the tasks are designed around Zimmerman’s self-regulated learning model. The submission date (time from deadline) and quality of the responses are the triggers for intervention.

I’ll admit the term “meta-learning assessment tasks” does sound rather alarming, and abstract to the majority of Academics / Faculty. However, within Colthorpe et al’s mode it breaks down into a relatively straight forward design.

They have four tasks evenly spaced throughout the course, each tasks contains 6 short questions and altogether they contribute around 12% to the overall grade.

The tasks include;

articulating the study strategies they had used in the past and identify hinderances to their learning

articulating strategies they may use to improve their learning and promoting effective study before an exam (mid semester)

reflecting on strategies they used for their exam (mid semester) identify how effective and how to improve

A message I took from the article was the importance of the forethought stage with academic achievement. They suggested this is a critical stage for success as it concerns organization and planning, motivation, beliefs and personal goal setting.

So not wishing to critique the whole article (I leave that to you), it would be good to consider how might a faculty/academic need to change practice in light of this article and my previous posts around assessment patterns. I’d suggest these ideas could be easily incorporated within curriculum learning models. For instance, the inclusion of non-discipline short answer question is easily accommodated within the VLE quiz engine, and their existing summative assignment diet. Therefore, apart from the allocation of marks for completion, it would be relatively straight forward to implement.

The use of marking rubrics should allow a quick assessment by the faculty/academics. The outcome is the intervention decision is informed by;

have they “successfully” completed the low stakes assessment?

was their submission “timely”?

did their submission demonstrate the “attributes of a self regulated learner?

The words in “” need to be further defined with the institutional and course culture, and the threshold levels need to be informed from analyzing the data.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s