The kinds of manipulation and gamesmanship that researchers could employ to boost their research ratings under the system set to replace the research assessment exercise have been outlined in frank detail to Times Higher Education by a number of senior academics.
Under the forthcoming Research Excellence Framework, which is out for consultation until 14 February, research quality in the sciences will be judged largely on the basis of the number of times an academic's work is cited by his or her peers.
One potential technique to maximise citations could harm young researchers looking to climb the citation ladder. Senior scientists could manipulate publication procedures used by their research groups to prevent first and second-year PhD students being added as secondary authors to group publications, a practice seen as affording students an early career boost. They could later be used as lead authors on group work on the understanding that they cited earlier papers from the group where their names did not appear. The practice would circumvent the Higher Education Funding Council for England's plan to exclude "self citations" - where academics cite their own earlier work - and would allow senior researchers to improve their citation counts substantially.
"Any system that undermines the aspirations and ambitions of our very best early-career researchers would be a step backwards," said one pro vice-chancellor.
Another senior academic said universities might introduce institutional publication policies to ensure researchers achieved maximum citations. "If Hefce chooses to harvest all papers produced from an institution, the cat is really among the pigeons," said the source. "Institutions will have to start policing every paper that leaves the institution destined for publication."
But behaviour that could hinder young researchers is just the tip of the iceberg, according to others.
"Citation clubs" - cartels of colleagues who do deals to cite each others' work - may become increasingly common. Observers also predict a boom in the number of spurious or sensational papers that are more likely to grab attention and citations.
"Lots of people quote bad science," one pessimistic senior researcher said.
"Citation clubs are already operating in the US," said one head of department.
"A lot of academic work is already based on mutual admiration and back scratching, and that will intensify," said another anonymous source.
Concerns that game-playing within the REF will undermine collaborative and interdisciplinary research are also being expressed.
"It is much better to ensure that joint papers aren't published so that the chances for citation by a group working in the same area are increased," said one professor.
Concerns are particularly acute within the engineering community, which Hefce acknowledges is not well covered by the Web of Science database that it intends to use to count citations.
"The one thing you can absolutely guarantee is that people will skew behaviour in order to make the best out of whatever metrics are in place," said Sue Ion, the vice-president of the Royal Academy of Engineering. She has been working on the society's submission to the REF consultation.
"If citations become the be-all and end-all - and Hefce has never said they will - then academic groups will look at where they should be publishing. The worry is that rather than doing collaborations with small to medium enterprises and industry that may not result in any publications, they will try to do detailed work to report in top-flight journals."
Dr Ion added that the society was looking at other metrics measures that might be introduced to measure interdisciplinary and collaborative research but that "light-touch peer review had a fair bit of attraction". However, she said, there was no "one-size-fits-all" solution for all of engineering's branches.
Among the more colourful suggestions offered by researchers as to how to improve citation counts under the REF is the use of more colons in the titles of papers.
Times Higher Education was referred by one academic to a paper by an American academic, Jim Dillon, "The emergence of the colon: an empirical correlate of scholarship", in the journal American Psychologist in 1981. It records a high correlation between the use of colons in the titles of published academic papers and other quality indicators, including citation counts.
"We might follow this to include at least two colons in the titles of our papers so as to ensure the highest recognition," said the academic, adding that the downside might be a mass breakout of what Dr Dillon might call "colonic inflation".
"I understand that it is extremely painful," he said.
Hefce says: "The main incentive for researchers will be to publish work that is recognised as high quality by peers and becomes highly cited by the international academic community. Short-term game-playing is far less likely to achieve this than responsible strategies to nurture the talent ... and for publishing work that enhances the group's international reputation." But more mischievous academics have suggested some ways to maximise one's rating:
- Do not cite anyone else's research, however relevant it is, as it boosts others' citation counts and increases their funding;
- Do not publish anything in journals with low citation rates, a group that includes lots of applications-based journals, as it will lower your citations;
- Do not do scientific research in fields not yet well covered by Thomson Scientific database, as your output won't be visible;
- Do not report negative results: they are unlikely to get cited;
- Join a citation club.