The rules for the UK’s next research excellence framework are finally in place, after a year of consultation.
The assessment, which will be published in 2021, will be used to determine the allocation of quality-related research funding, and will play a key role in forming the academic reputations of universities and their departments.
Following intense debate over issues including staff submission, the “portability” of research outputs and the role of impact in determining scores, many sector bodies and commentators have welcomed the changes proposed by the UK’s funding bodies as representing an improvement on the last REF, which was conducted in 2014.
The changes largely follow the recommendations set out last year in an independent review led by Lord Stern, who was then the president of the British Academy. So, how will the exercise work in practice?
Staff submissions
In the run-up to REF 2014, universities were able to choose which researchers they submitted, meaning any weaker faculty members or those perceived to be underperforming could be omitted from the exercise – a controversial move that risked putting individuals’ reputations at stake. For REF2021, the funding bodies have ruled that all staff employed on the census date of 31 July 2020 with “significant responsibility” for research must be submitted. Representative bodies including the University Alliance, Russell Group and MillionPlus have welcomed the decision, and it is hoped that the move will give a “more rounded” view of institutional performance. But critics argue that universities now have even more of an incentive to lay off researchers that they do not want to submit to the assessment.
Each participating staff member will be required to submit a minimum of one and a maximum of five outputs, with an average of 2.5 outputs for each full-time equivalent researcher. This differs from the 2014 exercise, for which each participating researcher was expected to submit four outputs.
The funding bodies say that contractual status can be used to identify academics with “significant responsibility” for research in the majority of cases and that, for “many submissions”, all academics will meet this criterion and the minimum contractual basis of 0.2 full-time equivalent. Responding to concerns that contract status alone may not accurately identify such staff in all disciplines or types of institution, however, the funding bodies have confirmed that there will be an alternative route available for including staff, whereby institutions can implement their own process for identifying those with significant responsibility for research. This process, which may include an assessment of the proportion of each academic’s time that is allocated to research, would have to be consulted on with staff and documented.
As in previous cycles, very small academic?units (departments with fewer than five full-time staff members) may request to be excluded from submission, but only in “exceptional circumstances”. The councils also say that staff will be allowed to be submitted “without the required minimum of one output where certain exceptional individual circumstances have affected their ability to meet the requirement”. There’s no escaping the REF if you die during the cycle – a university would still be eligible to submit your outputs.
Portability
In previous years, staff members moving institutions mid-cycle carried all their written and published work with them, meaning that their new employer could snap up all the credit in the subsequent REF. This led to a highly competitive “transfer market” in the run-up to the assessment, something that funding bodies say they aim to make a thing of the past.
While Lord Stern recommended that credit go to the institution where the work was “demonstrably generated”, the proposal was not welcomed by universities, and was a source of particular concern for early career researchers, who feared that being unable to take their publications with them would make them less attractive to prospective employers. In what appears to be a compromise, funding bodies have opted for a transitional approach, ruling that full credit will go both to the institution employing the academic on the census date and the university where the output was demonstrably generated.
This should allow for greater freedom of mobility for academics and give universities less incentive to hire and fire simply for the purposes of the assessment.
While the move has been largely welcomed, some regard it as a less than perfect solution.
“While this new rule is fairer, there is still a big drawback there in double counting,” said Yuan Ju, professor of economics at the University of York, speaking in a personal capacity. “Hefce [the Higher Education Funding Council for England] says this won’t be a major problem, but I think this will certainly vary between university departments.
“Take for instance economics researchers – they are very mobile and they can easily get more than one job during the REF cycle. Double counting will amount to grade inflation and it’s very unfair. It will present an incorrect measure of the quality of the sector.”
Instead, Professor Ju argued, assessors should consider giving half a credit to each institution. “This would give a true score without affecting the mobility of staff,” he explained.
Impact
Submissions for REF 2021 will include one impact case study, plus one further case study per up to 15 full-time equivalent staff submitted (as opposed to one per 10 in 2014), for the first 105 full-time equivalents submitted. After this, the requirement will decrease to one further case study for every 50 full-time equivalents submitted.
The latest ruling follows the funding bodies’ announcement that impact would account for 25 per cent of a university’s score in the next REF, up from 20 per cent in the previous exercise.
Pam Tatlow, chief executive of MillionPlus, said that this was “a step in the right direction, both in terms of increasing the value of the weighting, but also seeking to broaden the definition to recognise a wider range of impacts”.
Others have expressed concerns about the increased focus on impact for this cycle, arguing that rather than reducing burden – as promised by Hefce – increased case studies could detract from the focus on research excellence and cause some smaller departments to struggle.
“If you think about it, a department with up to 45 staff would have to submit four case studies, while an institution with 16 staff will need to submit three,” said Professor Ju. “I think this is really challenging for the smaller department and in that sense I find that rule a little bit unfair.”
Gaming the system
Many of the funding bodies’ changes have been aimed at reducing “game-playing” by universities and giving a fairer picture of institutional research performance. So, how have they done?
“Hefce and the other funding bodies have struck a reasonable balance,” said Graeme Reid, chair of science and research policy at UCL. “They have stopped the worst game-playing without imposing too much of a burden on institutions.?Now, of course, the community will look for ways of gaming the new arrangements – that is unavoidable.”
One concern is that universities may manipulate the full submission rule by changing staff contracts from “teaching and research” to “teaching and scholarship” – taking perceived weaker candidates out of the running as before.
“Currently this REF will certainly reduce the potential for gaming but it won’t completely solve it,” said Professor Ju. “The departments or universities will of course try to look at changing staff contracts…This could have job implications and other consequences for the sector.”
But Alex Bearman, a partner at law firm Russell-Cooke, said that “attempting to impose unilateral changes to employment contracts in order to exclude certain staff members from the REF is likely to be fraught with difficulty”.
“It is quite possible that affected employees would argue that forcing them to accept such changes would entitle them to resign and claim constructive dismissal, particularly if it can be shown that inclusion in the REF has significant value for the individual in terms of academic standing and future career prospects,” he said.
Any tribunal claims that result would be likely to expose the universities’ attempts to manipulate the system, potentially causing significant harm to their reputations, Mr Bearman noted.
“My personal feeling about the new rules is that they are a clear improvement on the last round,” added Professor Ju. “I think Hefce has responded fairly to Stern’s recommendations, and this will make for a fairer assessment. Requiring all research staff to submit is a good thing, but this will not solve the gaming problem.”
