Source: Rex
Read all about it: some argue that the ABS journal guide is ‘intellectually vacuous and empirically false’, while others defend it as a ‘robust’ tool to be used ‘holistically’
With the dust from the research excellence framework having barely settled, the march towards greater measurement and ranking in higher education is clear for all to see.
On this path, one of the most hotly debated innovations has been the emergence of journal lists, which assign ratings to whole periodicals and not just to individual articles.
There are two goals. One is to help researchers to make informed decisions about where they should seek to have their work published. Which journals have the highest standards, largest readerships and most frequently cited papers?
The other is to aid heads of department and other leaders in their attempts to assess the performance of academics, particularly in broad disciplines with numerous sub-specialisms.
For critics, this second mission has become the primary function of journal lists, making them the key consideration in decisions over hiring, promotion and salary review, and leading to researchers being judged largely on where they publish rather than what they publish.
At the front of this debate has been business and management studies. The Association of Business Schools’ , published on 25 February, assesses the quality of 1,401 publications worldwide, drawing on citation scores and the judgements of leading researchers, among other criteria.
It gives the coveted 4* “journal of distinction” title to 33 publications, while 85 are rated 4 – meaning that they contain the “most original and best executed research”.
Some 312 are rated 3, characterised as periodicals of “original and well executed research”, with 481 scoring 2 for “original research of an acceptable standard” and 490 in the bottom tier of 1, with work of a “recognised, but more modest standard”.
The number of journals ranked between 4* and 3 has increased significantly since the last edition of the guide was published five years ago, from 324 to 430. But since the 2010 version rated only 823 titles, this represents a marked fall as a proportion of the total, from 39.4 per cent to 30.7 per cent.
Angus Laing, chair of the ABS and dean of business and economics at Loughborough University, argued that the results were heartening.
“What we are very encouraged by is the number of journals that have been rated in the top bandings; that reflects well on the richness, depth and breadth of the business school community,” he said.
“What I think is important is that there is increasing evidence that UK and [continental] Europe-originated journals are increasingly sitting up alongside American journals.”
The guide will prove particularly useful for early career researchers considering where to seek publication, and for more experienced academics stepping across disciplinary divides, Professor Laing said.
But for Dennis Tourish, professor of leadership and organisation studies at Royal Holloway, University of London, the guide is “intellectually vacuous and empirically false”.
There is a “far from perfect” correlation between the rating of a journal and the quality of its articles according to exercises such as the REF or the 2008 research assessment exercise, he said. And that is after some schools used the guide to decide which outputs to submit.
In the REF, roughly half the business and management outputs considered received identical scores to the 2010 ABS grade of the journal in which they appeared.
Professor Tourish argued that, since researchers are now put under pressure to publish only in 3- or 4-rated journals, the guide acts as an essentially conservative force.
The guide has traditionally been dominated by American titles, which typically emphasise a functional view of the discipline and have declined to explore the impact of the financial crash, he said.
Newer journals find it harder to attract good papers and establish themselves as a result of the guide, according to Professor Tourish.
The guide’s editors are clear that any title not in the 2010 list is limited to a maximum rating of 3 in the latest edition, “other than in exceptional circumstances”.
“People are being shoehorned into publication practices which are not consistent with their values and interests,” said Professor Tourish.
“They have to conform to the publication practices and topics that are considered relevant by elite journals. It limits the scope of academic enquiry in a way I think is harmful.”
This view is echoed by John Mingers, professor of operational research and systems at Kent Business School. He recognises the value of the guide, particularly when looking across a broad discipline, but is concerned about how “hegemonistic” it has become.
“Young researchers who might come along with bright new ideas get put off from being able to publish them,” Professor Mingers said. “I think it has an impact on how much new and groundbreaking research can be done, if we are following fixed pathways.”
For Geoffrey Wood, co-editor of the 2015 edition and professor of international business at Warwick Business School, these complaints are about how the guide is used, not the guide itself.
Good research can be found in all types of journal but does tend to be clustered in certain titles with high standards of peer review, he said.
Identifying these clusters is important, Professor Wood argued, in order to allow academics to plan their careers with confidence and to encourage lecturers to push themselves harder.
In the digital age, the guide acts as a form of Kitemark of quality, protecting young researchers from online journals that may seek to exploit them financially or intellectually, he added.
What is perhaps most important for Professor Wood, however, is the methodology of the guide, which involves consultation with academics and subject associations across a wide range of disciplines.
Prior to the emergence of the ABS ranking, first published in 2007, lists produced by individual business schools held more sway, but these often reflected the interests of the school in question, or even where its leaders published, he said.
“If this is going to happen, it’s better that it happens in a civilised manner, rather than in a barbarous fashion,” Professor Wood said.
Professor Wood agreed with Professor Laing that the information in the guide should not be considered in isolation.
“This is one piece of information – a very robust piece of information – that can be used holistically in making a judgement, either when choosing where to publish or when assessing how an academic’s career is progressing,” Professor Laing said.
“It is certainly not planned as something that should be driving decisions narrowly and in a mechanistic manner. Any dean who does that is a fool.”
