Source: Alamy

Like pulling teeth? Some say costly REF measurements should be scrapped

Allocating an institution¡¯s quality-related research funding on the basis of its research council income would not significantly change outcomes, a government-commissioned report has concluded.

The report - by the Centre for Business Research, based at the University of Cambridge, and the UK Innovation Research Centre, based at Cambridge and Imperial College London - examines the effect of the UK¡¯s dual funding system for research on knowledge exchange and commercialisation.

It notes a ¡°strong¡± correlation between the amount of research income that institutions earn from the various sources.

ÁñÁ«ÊÓƵ

Some observers have long argued that the costly and labour-intensive research assessment exercise, and its successor, the research excellence framework, should be scrapped and QR funding allocated based on each institution¡¯s research council income.

The report, , notes that the 2008 RAE¡¯s cost to the sector has been estimated at nearly ?50 million.

ÁñÁ«ÊÓƵ

The researchers examined what would have happened if mainstream QR funding in 2009-10 and 2010-11 had been distributed on the basis of each institution¡¯s share of research council income earned between 2002-03 and 2008-09. They found no average difference between the actual and hypothetical allocations.

The QR allocation of the biggest losers would have slipped by 0.9 per cent, while that of the biggest winners would have risen by 2.6 per cent. Most of the winners would have been the ¡°top handful¡± of institutions because of greater concentration in research council funding.

In 2010, the top 10 per cent of institutions claimed 53 per cent of QR income and 64 per cent of research council income. However, from 2012-13 QR funding became more concentrated after 2* research was removed from the formula. As a result, the mainstream QR allocations of England¡¯s five biggest earners - Imperial College, University College London and the universities of Oxford, Cambridge and Manchester - slipped by 0.9 per cent, compared with an overall drop of 3.2 per cent.

But Mark Burnley, senior lecturer in the University of Kent¡¯s School of Sport and Exercise Sciences, said that the study¡¯s statistical interpretation and presentation of data - such as its lack of correlation plots and error bars - was ¡°so poor as to render the information almost meaningless¡±.

ÁñÁ«ÊÓƵ

He said that using research council income to determine the allocation of QR funding would be ¡°fundamentally unfair¡± because it would work against disciplines that derived higher proportions of their income from other sources. It would also disadvantage newer institutions or departments that had not yet won any research council income.

Giles Carden, director of strategic planning and analytics at the University of Warwick, said a previous Treasury study had also shown a high correlation between research council income and RAE scores. Allocating QR cash on the basis of the former made sense ¡°if the government is simply interested in [finding] the simplest method possible¡±, he said. But, he continued, the REF also provided a means to assess the quality of the outputs of UK research and had spurred universities to focus on maximising their research excellence.

paul.jump@tsleducation.com