European universities continue to rely heavily on publication metrics – in particular, the much-criticised journal impact factor – when assessing academic performance, a study suggests.
However, of a survey of about 200 institutions by the European University Association indicate that one of the main obstacles to reforming research assessment is resistance to change from academics themselves.
The survey, to be published in full later this month, says that research publications and attracting external research funding were rated most highly when universities were asked which type of work mattered most for academic careers, selected as being important or very important by 90 per cent and 81 per cent of respondents respectively.
Asked how academic work was evaluated for career progression decisions, publication and citation metrics ranked top, selected as important or very important by 82 per cent of respondents.
And, despite criticism of journal impact factor – a citation-based evaluation of the periodical in which an academic’s work is published, not an assessment of the impact of the paper itself – three-quarters of institutions said that they used it to evaluate staff performance, more than any other metric. Academics argue that journal impact factor is an unfair metric and can be open to manipulation.
Seventy per cent of respondents to the EUA survey said that they used academics’ h-index, a measure of productivity and citation impact, as part of their assessments.
Bregt Saenen, the EUA’s policy and project officer, said that the widespread use of journal impact factors was “one of the most disappointing results from the survey”.
“The quality of a journal article should be assessed based on the merit of the research/article itself, not on the reputation of the journal in which the article is published,” he said.
Dr Saenen said that universities needed to move towards “a less limited set of evaluation practices to assess a wider range of academic work”.
With this in mind, Dr Saenen said that it was “encouraging” to see other areas being regarded as important or very important by respondents, such as research impact and knowledge transfer (68 per cent), supervision (63 per cent) and teaching (62 per cent).
Seventy-four per cent of respondents said that qualitative peer review assessment was an important or very important factor in career progression decisions.
However, asked what the main barriers to reviewing research assessment were, 33 per cent cited resistance to reform from academics themselves. This was one of the most popular responses, alongside the complexity of reform (46 per cent), lack of institutional capacity (38 per cent) and concern over increased costs (33 per cent).
Dr Saenen said that key barriers for universities were likely to be “accountability to research funding organisations and governments in their approach to research assessment, as well as the influence of the competitive environment in research and innovation”.
Reviewing research assessment procedures “is a shared responsibility and will require a concerted approach”, he said.
Print headline: Citation metrics reign supreme
