ÁñÁ«ÊÓƵ

Impact broke the academy. Only a culture change can fix it

<ÁñÁ«ÊÓƵ class="standfirst">Linking impact to funding is breeding mistrust, apathy and unrest. Researchers must be free to do the work they find most meaningful, says Mark Reed
May 11, 2022
car breakdown
Source: Getty

I will be looking at this week¡¯s Research Excellence Framework (REF) results from a unique vantage point.

As well as having experienced first-hand the weight of responsibility and pressure to achieve impact success before the submission date, I also trained or advised every Russell Group university ¨C and the majority of other institutions ¨C over the last REF period. And I saw the underbelly of the impact agenda, as colleagues scrambled to construct, evidence and polish their own narratives.

For every success story that we hear on results day, there will be another untold story of an impact that went wrong. Some academics attempted to speed up, push through or skip vital testing stages in an attempt to get results before the REF deadline. Others claimed impacts that wouldn't have happened without the help of external organisations or junior colleagues who weren¡¯t given credit because there wasn¡¯t enough room in the allotted five pages. Still others requested or drafted testimonials from stakeholders who didn¡¯t recognise the impacts they were being asked to corroborate and sometimes couldn¡¯t even remember the researchers.

I have experienced these impact bounty hunters first-hand, too, as research lead for an international conservation charity. Despite our having very limited resources, researchers regularly expect us to write last-minute letters of support for proposals we don't have time to review properly. It is assumed that we will contribute to academics¡¯ projects without charging for our time, working alongside a researcher who is paid significant overheads on top of a salary that could easily be greater than the combined salaries of our entire team. Sometimes we respond to invitations for workshops we are unlikely to benefit from just so we are there to help put out the fire when researchers inflame conflicts in our networks.

ÁñÁ«ÊÓƵ

Such experiences have contributed to the growing erosion of trust between researchers and society. How did it get like this? How did we lose our way? The answer is simple: we took our eyes off our original purpose to focus on the prize that impact now is.

It is not a coincidence that the impact agenda has led to so many negative unintended consequences in the UK. Nowhere else has impact been so directly linked to funding, and hence to so many extrinsic incentives, from rankings to promotions. This might have been necessary to get people to engage in the early days, but these incentives are breeding not only mistrust but also apathy and unrest.

ÁñÁ«ÊÓƵ

In a 2020 of?more than 4,000 UK researchers, 75 per cent said their creativity was being "stifled" by the impact agenda. Over half felt pressured to meet REF targets. Research culture is broken, and the impact agenda is part of what broke it.

So how can we fix it? Some have advocated refusing to submit to demands for impact case studies. I have argued that we need to weaken the link between funding and impact by . But instead of starting with the system, which tends to lead to one-size-fits-all solutions, we should instead focus on creating a healthy research culture.

Cultures are co-created by people through interaction with others who share compatible values and aspirations; they can't be imposed from the top down. And that means that a healthy culture will always be composed of multiple subcultures, constantly evolving with the ebb and flow of new researchers and collaborations.

The role of the institution in this more inclusive, bottom-up model of culture change is to facilitate individuals and teams to pursue the work they find most meaningful. Not everyone has to generate impact, and not every subculture will have impact as a focus. However, impact will be one of many emergent properties of a system that supports ethical, robust and action-oriented research, promotes engagement with those who might benefit from research, and enables researchers to focus regularly on their core purpose (whatever that may be).

ÁñÁ«ÊÓƵ

The regularity and timing of work on these priorities is more important than the overall amount of time spent, so I am not advocating for an individualistic culture in which we abandon our responsibilities and team members. By designing spaces to enable creative collisions with people from industry, policy and the like, as well as with other researchers, it may be possible to help researchers find deeper purposes for their work, as they discover new ways to benefit society.

You might feel broken by the system you work in, but it is possible to take small steps to create a culture you can belong in. The hope must be that unexpected new ways of working emerge and spread from person to person and group to group. The emergence of a diverse, authentic and values-driven culture will inspire the creative thought that the world needs so badly right now.

Mark Reed is professor of rural entrepreneurship and director of the Thriving Natural Capital Challenge Centre at SRUC. He is CEO of Fast Track Impact, which offers health resilience training and coaching. His latest book, Impact Culture, is published by Fast Track Impact. Most of it is available open access, with free resources and a year¡¯s worth of free training and events, at .

Register to continue

Why register?

  • Registration is free and only takes a moment
  • Once registered, you can read 3 articles a month
  • Sign up for our newsletter
Register
Please Login or Register to read this article.
<ÁñÁ«ÊÓƵ class="pane-title"> Related articles
<ÁñÁ«ÊÓƵ class="pane-title"> Reader's comments (5)
Impact is not the be all and end all but it is very important. If the society has funded you, it is only pertinent to ask how your work has benefitted the funder. What is find ridiculous is ex-panel members selling their services to institutions to facilitate the most ridiculous shadow REF exercise. Everyone knows what the purpose of that exercise is. I hope the next REF stipulates that ex-panel members are not allowed to offer their services in this manner and not create a market that can have all sorts of unintended consequences. No one is forcing anyone to do impactful work. If you want to do purely academic work, do so. I hope that Environment and Impact have even more weight in the future REFs.
You are forgetting also it is a lotto. So a person might right on some intellectual property issue and by chance a year later it becomes and issue in parliament and their research ideas are adopted into law giving them 4* impact. Meanwhile, their colleague who has written on the law of trusts might not see parliament deciding ever to address the issue and thus no matter the quality of the research, it will not be impactful.
There is the saying: if a measurement becomes the target, it is no longer a good measurement. This applies to the REF, too. Firstly, it is impossible to know what the impact of a given fundamental research is ahead of time. Yet, we have to come up with great amount of bullshit/science-fiction to get funded. If researchers were accountable for the actual impact of their research, most of those who get funding would go to jail for fraud. And then there is unfunded research, that people do at night, weekends, etc, which cannot be funded, because the quality of bullshit was not sufficient to convince peers. These unfunded project then will have their impact. This is how ridiculous the impact of maths is: ?588 return for each ?1 spent (as per IMA estimation), because a) we do not get funding (lack of perceived impact) b) if we do get funding, it is minimal. In comparison Engineering produces ?88 for ?1 invested, Physics ?31.
Impact should be no more than 10% of the criteria for distributing funds. For example, 54% of Surrey Law School's outputs were rated 4* while only 35% of Oxford Law Faculty's outputs were rated 4*. Surrey also produced more 3* outputs than Oxford, yet because of impact Oxford ranked 14 in overall in Law for REF, while Surrey ranked 43. This means Oxford will get far greater funding for producing lower quality research simply because it has the brand to get the invites to select committees etc to have impact. It is grossly unfair that a institution with a substantially greater proportion of 4* outputs gets less funding simply because it has less social ties to the lawmakers to get that research being considered in lawmaking decisions. Environment is also another unfair criterion, because the big name universities attract tenfold the number of PhDs and postdocs making it for smaller institutions to compete. The real focus needs to be on the ranking for 4* outputs: in that Surrey Law School was 6th, while Oxford Law Faculty was 23rd. Thus, if Oxford's reserach is of a lower quality proportionately, why is it having far greater impact?
In some disciplines the journals are controlled by US schools and unless you are from the US or have US coauthors nearly imlossible to get published in top 5 journals. REF may indeed rate your work as 4* even if it is out of the top 5 but we dont know what the individual scores are from the REF. So what happens on the REF matters for nothing for the individual academic who are still judged by where they publish. Academia should be more just publishing. Agree with the point about impact being long term and the REF does allow for that. Environment is also just as important.
<ÁñÁ«ÊÓƵ class="pane-title"> Sponsored
<ÁñÁ«ÊÓƵ class="pane-title"> Featured jobs