In the recently published 2021 Research Excellence Framework, 84 per cent of the research submitted was deemed either “world-leading” or “internationally excellent”.
This is what the REF, in its various guises, was meant to do: provide incentives by steering scarce funding to centres that demonstrated greater excellence. This process of episodic nudges, however, has introduced large-scale changes to the structure of academic disciplines in the UK that may pose problems in the future, for both individual scholars and the sector at large.
Slowly but surely, UK disciplines have been moved in the direction of greater intellectual homogeneity. But study after study demonstrates that serendipity and diversity are critical for innovation and the advancement of knowledge. Homogeneity constrains our imaginations and our ability to find solutions to collective problems.
The evidence for convergence is more than anecdotal. In my forthcoming book, : How Research Evaluations Transformed the British Social Sciences, I?look at a unique longitudinal dataset containing information on the careers and publications of more than 16,000 academics working in the UK between the early 1980s and 2018. This allows us to trace changing patterns of employment and mobility since the inaugural “Research Selectivity Exercise” in 1986.
I focus on the “softer” social sciences of anthropology, economics, politics and sociology owing to their greater flexibility, adaptability and organisational diversity. While many experimental sciences require vast investments necessarily anchored to global theoretical standards, social science research relies on less costly investments and speaks to local concerns, typically involving smaller teams (often individuals).
Some of the changes we already knew about. Article productivity has increased; women are more likely than men to change jobs and to do so laterally (without experiencing promotions); and citation is heavily skewed by gender. The dataset, however, allows for more sophisticated analyses. Critically, combined with information about scholars’ publications, it allows sophisticated computational text analysis and classification to get a sense of how changes in career structures were coupled with changes in knowledge.
I find evidence that scholars were more likely to move away from jobs where their contributions were close to those of their colleagues, and from departments that were very distinct from the national norm. The direction of movements was also notable: scholars would find jobs in situations that made them conform more closely to disciplinary standards. The result isn’t just diminished job stability (itself a valuable social good) but also increasingly similar departments and research output.
A computational measure of departmental diversity, for example, shows that sociology departments in the UK today have the same thematic diversity as economic departments had in the 1980s. This drop in diversity reflects a reduction in centres of excellence that deviate from disciplinary norms but that can be the source of critical innovations (think, for example, of Birmingham’s Centre for Contemporary Cultural Studies, established in the 1960s but closed in 2002).
Further evidence of these shifts is provided by computational analysis of changes in word use. The issue is not simply a loss of organisational diversity, but also of theoretical, epistemic plurality: over the past three decades, the use of concepts in all social sciences (except anthropology, for which adequate data?were lacking) has become increasingly standardised.
By itself, this does not explain why scholars changed jobs or how research evaluations factored into their research decisions. But my follow-up interviews were telling. Scholar after scholar noted that while their research was not driven primarily by the imperatives of REF, it was driven by how their immediate managers interpreted REF rules and strategy. While the REF does not use journal rankings or quantitative indices, for instance, some departments ask their academics to create lists of journals deemed adequate for their publications. In others, academics tacitly accept these lists through their disciplinary affiliations. And in yet others, managers dictate what is valuable based on their own interpretation of what world-leading research ought to be.
Moving away from this dynamic is difficult, not least because the mechanics of these evaluations (which are, after all, mostly managed by colleagues) reflect shared, if inadequate, notions of scholarly merit. Indeed, even in systems that do not have comparable exercises to the REF, the pursuit of “world-leading” research is still a preoccupation, especially in the top?research-intensives. That is certainly true in the US, whose greater breadth is mostly a product of its greater size; intellectual diversity is lower in the top universities – many of which, indeed, are seen to define what “world-leading” looks like.
But while the quest for excellence might have had wide support when academic work provided stable and rewarding employment, the goodwill necessary for playing the REF game is fast evaporating. More than two years of a devastating pandemic has laid bare inequalities within institutions and the sector at large. Amid department closures and reorganisations, not to mention the scandalous treatment of pensions, even are increasingly leaving the sector.
If a future exists for research assessment, targets for excellence must be pegged to the actual everyday experiences and barriers faced by UK academics. They must not only look outside, but also within, to what is possible. Without such recognition of the importance of building a hospitable profession, any measure of excellence will ring hollow.
Juan Pablo Pardo-Guerra is associate professor in sociology at the University of California, San Diego. : How Research Evaluations Transformed the British Social Sciences is forthcoming from Columbia University Press.
Print headline:?Research evaluations are narrowing UK disciplines
