A shift?against “publish or perish” research culture is building with the release of a European agreement on how research assessment should be reformed.?
The sets out the principles, core commitments and broad timelines that funders and institutions can sign up to if they want to make their assessment of teams, projects and researchers fairer.
Signatories agree to recognise the diversity of contributions to research, base evaluation mainly on qualitative measures, abandon journal-based metrics, avoid the use of rankings and commit resources to reform.?Each will be expected to submit an action plan with milestones by the end of 2023 or within one year of signing the agreement.?
“It is action-orientated, so it’s not just a declaration. The aim here is not just to stop at the agreement but to put together a coalition,” said Stephane Berghmans, director of research and?innovation at the European University Association, who helped lead the drafting of the text.?
More than 350 organisations from over 40 countries have met in three assemblies to give their take on the text. Influential players, such as the German Research Foundation, have pushed back against setting concrete targets.
“It has raised reluctance for some. The fact that it sparks all these reactions is a good sign that we are proposing a firm action plan,” said Lidia Borrell-Damián, secretary general of the funder body Science Europe, who also led the drafting.?
Letting institutions work out how they want to reform allows a diversity of approaches between countries, institutions and disciplines, she said:?“Our aim is not to replace one absolute system with another absolute system. Surely it will not be one predominant model, but several co-existing models, and that’s precisely what we want.”?
Those who sign will meet regularly from the autumn to trade advice on how best to update assessment. “They can pose the tricky questions, and how those that are more advanced have solved these,” said Professor Borrell-Damián.?
Dr Berghmans said reformers would use the EU’s ERA (European Research Area) Forum to point out pan-European barriers to all EU ministries, while smaller working groups could lobby specific governments on national snags.
The agreement “goes one step further” than 2015’s “largely symbolic” Leiden Manifesto, which set out 10 principles for measuring research performance, according to Ludo Waltman, who studies quantitative science at Leiden University and helped write the manifesto.
He said reform had to be coordinated across the continent for researchers to move freely between jobs and know they will be judged by similar standards.?
Backing from the European Commission means organisations “kind of need to decide whether they want to be part of” reforms or not. “Those that didn’t like the Leiden Manifesto simply ignored it,” he said.?
However, Professor Waltman questioned whether the commitments to abandon journal-related metrics and step back from rankings would really be possible without deeper changes to research culture.
“As long as we maintain that way of thinking about publishing and the prestige economy around it, we may question whether it is realistic to move away from impact factors,” he said, adding that it was hard to see how universities could keep the influence of rankings away from assessment.
Other challenges will be national. Ismael Ràfols, a researcher at the University of Sussex who also wrote the Leiden Manifesto, said countries that depend on centralised assessment, such as Italy, Spain or Poland, would find it particularly hard to wean themselves off scalable publication metrics.?
“The debate is whether the governments have the willingness or the capacity to decentralise, because there is no trust in organisations, due to the nepotism that was common 30 years ago before centralised systems were implemented,” he said.
Sarah de Rijcke, another Leiden Manifesto drafter, welcomed references to scientifically judging the best forms of?assessment and said she expected minimum standards to emerge in the coming years.
“After a few years, the coalition will want to bring emerging practices together and will want to aim for some minimal standards...that will hold across participating organisations. Standardisation where possible, diversity where necessary.”
Print headline: Euro pact to counter ‘publish or perish’ ethos
