Moving away from ¡°tried-and-tested¡± methods of evaluating research excellence via outputs-based peer review towards more experimental ways of?judging academic culture is?causing ¡°nervousness¡± and ¡°uncertainty¡± at?leading UK?universities, a?Russell Group leader has admitted.

Under?proposed rules for the 2028 Research Excellence Framework,?just 50?per cent of an institution¡¯s score will be judged on its research outputs, down from 60?per cent in 2021 and 65?per cent in 2014. Of this 50?per cent, at least 10?per cent will be decided on an institution¡¯s self-described disciplinary contribution, potentially pushing the direct weighting of outputs close to 40?per cent. Under the?revamp announced on 15?June,?¡°people, culture and environment¡± will be worth 25?per cent of an evaluation, the same as ¡°engagement and impact¡±.

Downgrading research outputs, however, had been raised as a concern by senior university leaders, said Colette Fagan, chair of the Russell Group¡¯s research pro vice-chancellors¡¯ group. ¡°In our early discussions, there were concerns about this reduction from 60?per cent towards 40?per cent,¡± said Professor Fagan, vice-president for research at the University of Manchester, who said there was ¡°some nervousness and uncertainty¡± about revising the assessment formula used to decide the allocation of ?2?billion in annual research funding.

¡°We want to measure research excellence in the best possible way, and assessing outputs and impact case studies is a tried-and-tested method,¡± continued Professor Fagan, who said little was known about the proposed disciplinary statements, even though Russell Group members ¡°broadly welcomed the direction of travel¡± for REF 2028. ¡°A lot will rest on the consultation on the disciplinary statement and how it relates to wider institutional assessment,¡± she added.

ÁñÁ«ÊÓƵ

That concern over the shift away from measuring research outputs was expressed more robustly by former ministerial adviser Iain Mansfield, now head of education and science at the Policy Exchange thinktank, who questioned whether having ¡°people judging if the ¡®culture¡¯ is ¡®right¡¯ for producing great research, rather than actually looking at the outcomes¡±, was the correct approach. ¡°No doubt they¡¯ll pat the institutions with the right buzzwords on the back,¡± said Mr Mansfield.

Gemma Derrick, associate professor in research policy and culture at the University of Bristol, said the shift away from measuring outputs would make the REF ¡°more forward-looking¡± rather than focusing largely on assessing past excellence.

ÁñÁ«ÊÓƵ

¡°It is also increasing the kinds of outputs submitted, so they are not just articles or books,¡± said Dr Derrick. ¡°It¡¯s a good step, even if it comes with risks, but the REF is mature enough to take some risks.¡±

Removing the need for?every research-active staff member to submit at least one output?could, however, increase game-playing by institutions, warned Dr Derrick. ¡°The inclusion of all staff in REF 2021 was widely celebrated ¨C I¡¯m not convinced why it¡¯s being abandoned,¡± said Dr Derrick.

James Wilsdon, professor of research policy at UCL, welcomed the ¡°positive package¡± of REF measures, but also warned that the downgrading of research outputs, allied with a shift to team-based submissions in which not everyone entered, could be unpopular with academics.

¡°Downgrading the significance of individuals could cause pushback because academics want to be connected to the REF ¨C now some will have outputs entered, and others won¡¯t,¡± he said.

ÁñÁ«ÊÓƵ

On the research culture statement, which will take a more rigid questionnaire approach, Professor Wilsdon said there was ¡°a lot of work to do¡± but agreed with the proposed new format. ¡°This isn¡¯t an exercise in storytelling ¨C it will combine a strategic overview of research strategy with responsible indicators?that relate to aspects of research culture and environment,¡± he said.

Kieron Flanagan, professor of science and technology policy at Manchester, also felt the shift away from evaluating research outputs was wise, even if it was a significant departure from earlier REFs.

¡°It¡¯s extremely reductive to see research as purely around outputs ¨C the REF has peer-reviewed outputs because it is easy and less controversial, but it should be prepared to do something different,¡± said Professor Flanagan. ¡°We don¡¯t live in the 1980s ¨C research is much more complex, dynamic and internationalised. The REF must reflect this.¡±

On the spiralling cost of the REF, which had an?estimated price tag of ?471 million?in 2021, Professor Flanagan said much of this expenditure arose because ¡°leaders want to feel in control¡± and therefore created expensive internal selection processes.

ÁñÁ«ÊÓƵ

¡°All universities need to do is hire good people, treat them well and let them get on with research, but today¡¯s managerial university leaders don¡¯t think like this,¡± he said.

jack.grove@timeshighereducation.com

REF 2028: key proposed changes

• Need to submit all research-active staff set to be scrapped, with institutions submitting an average of 2.5 outputs per researcher instead
• Outputs to make up 50 per cent of overall score, down from 60 per cent, with at least 10 per cent to be decided on statement outlining ¡°wider contribution to knowledge and understanding¡±
• Environment renamed ¡°people, culture and environment¡±, with increased weighting of 25 per cent and switch to questionnaire-style statements
• ¡°Impact and engagement¡± remains at 25 per cent weighting. Explanatory statement to make up between 20 and 50 per cent of score.