Ministers pressed ahead with plans for the teaching excellence framework despite having been warned that students were “not very enthusiastic” about the exercise and that the preferred metrics may fail to significantly differentiate between institutions.
Research released by the Department for Business, Innovation and Skills in the wake of the publication of the higher education White Paper has prompted claims that the government is intent on pushing through the TEF “come what may” and despite questions about whether it will accurately reflect standards in English universities.
The newly published documentation includes a on the subject of teaching quality conducted with students at 10 universities, led by David Greatbatch, a visiting professor at Durham University Business School.
This concludes that although most students agreed that they would have considered TEF data when applying to university, “many doubted” whether the information would have influenced their choice of institution.
“The students were, in general, not very enthusiastic about a TEF at the institution level, although there were those who thought it might be of some value,” the study says.
There was more support for running the TEF at subject level – something that is not now planned until the fourth year of the exercise.
Professor Greatbatch’s study includes a literature review that finds that there is a “striking” lack of agreement about what constitutes teaching excellence and that many researchers were “dubious” about the value of the kinds of output measures that are being considered as TEF metrics, such as graduate employment, for measuring quality.
BIS also published of the potential for using the results of the National Student Survey and the Destinations of Leavers from Higher Education study in the TEF, conducted by the Office for National Statistics.
This raises questions about the level of differentiation that may be found between institutions, highlighting that comparisons between the raw NSS data for providers were “not usually significant”, with the exception of a few outliers at either end.
The analysis goes on to suggest that it may not be possible to break down NSS and DLHE results by course or by other characteristics such as class or gender because of the small sample sizes.
Dorothy Bishop, professor of developmental neuropsychology at the University of Oxford and a critic of the TEF’s proposed methodology, told Times Higher Education that the apparent lack of engagement by BIS with the ONS work showed how ministers were “determined to bring this in come what may”. Professor Bishop said that the ONS appeared to be “dubious” about the reliability of NSS and DLHE data.
Paul Ashwin, professor of higher education at Lancaster University, agreed that the government was yet to articulate a strong case for why its chosen metrics were the right ones on which to measure teaching quality. But he argued that the decision to benchmark the metrics by factors such as entry grades and ethnicity meant that the results would be a lot more reliable than many existing league tables.
A BIS spokeswoman highlighted that a found that 93 per cent of applicants thought it was important to have reliable information about teaching quality in universities, but only 59 per cent were able to access such data. More than half of respondents said it was a concern that teaching quality varied across institutions.
“We are tackling variability in teaching standards across the sector and addressing the widely recognised imbalance between teaching and research, rightly recognising universities that offer the highest teaching quality,” the spokeswoman said. “It’s also important to note that as we implement the TEF, we will be moving to do so at discipline level.”
Print headline: Students ‘not enthusiastic’ about TEF, ministers heard
