Researchers at the University of Cambridge believe they are a step closer to cracking the measurement of students’ “learning gain”.
Three academics at the institution have developed a tool which uses a mix of survey and test questions to track the change in learners’ knowledge, skills and values during the course of their degree.
The exercise can be completed online in just over 20 minutes, potentially addressing one of the key criticisms of other measurement tools: that they take too long, and that, as a result, students cannot be persuaded to participate.
The tool also addresses concerns that US-designed assessments, such as the Wabash Study, are tailored for use with a liberal arts curriculum, and hence are not applicable in UK universities, since subject-specific degrees typically lack a common core of knowledge that can be assessed.
Testing of the tool, involving 4,782 undergraduates and postgraduates at 11 universities, has so far indicated that it has high levels of reliability.
Jan Vermunt, Sonia Ilie and Anna Vignoles, from Cambridge’s Faculty of Education, presented their evidence at the annual conference of the Society for Research into Higher Education on 6 December.
Their tool tracks students’ development across four key areas: cognitive, such as reasoning ability and structuring; metacognitive, including self-management and grit; affective, covering emotional and social engagement; and socio-communicative, primarily academic writing. It also asks students about their epistemological beliefs and their attitude to research.
This broad approach, covering areas which were selected following interviews with students and a literature review, should give a more holistic view of students’ development than many other learning gain measurements, which often focus on critical thinking skills, said Dr Ilie.
On average, students take 23 to 24 minutes to complete the questionnaire. They are asked to fill it in three times: the first two occasions five to six months apart, and the third time a year after the second. So far, the pilot cohort has completed the survey twice and, once the third wave is complete, assessment of the gains that students have made will begin.
However, based on the research conducted so far, the SRHE paper says that the Cambridge tool has “the potential to be used at scale in English higher education”.
Dr Ilie told Times Higher Education that the tool could be a useful way for universities to “explore their own practice” and to improve pedagogies and curricula.
The UK government has expressed interest in incorporating measures of learning gain into assessments of university performance as part of the teaching excellence framework, but Dr Ilie said it could be difficult to make comparisons between different disciplines.
“Regardless of what discipline you are in, you could make gains on these things, but potentially in a different configuration,” she said. “A simple one-to-one comparison where you put everything together in a ranking doesn’t make sense at this point.”
However, Dr Ilie said that the Cambridge measurement could be a “really important tool” for universities to identify the actions needed to improve teaching and learning, which can feed into TEF submissions.
