A proposed overhaul of Poland¡¯s research assessment system has been met with scepticism, with experts raising concerns about cost, bias and a potential negative impact on universities.

Produced by the Polish Academy of Sciences, the??encourage a move away from bibliometrics in favour of peer review and expert assessment. Funding, the academy advised, should comprise a ¡°basic subsidy¡± based on the number of employees and the ¡°cost-effectiveness¡± of the research unit, as well as supplementary funding that matches a percentage of external grants received and an ¡°excellence subsidy¡± determined by expert assessment.

Because of?cost and organisational limitations, only ¡°units that have already demonstrated their high quality¡± should receive expert assessment, the academy said. Overall, the academy said, the proposed system would be ¡°relatively simple and not susceptible to manipulation¡±.

Marcin Pa?ys, a former rector of the University of Warsaw and chair of Poland¡¯s Central Council of Science and Higher Education, welcomed the academy¡¯s emphasis on expert assessment over bibliometrics. A reliance on bibliometrics, he said, can incentivise researchers to churn out papers, regardless of quality.

ÁñÁ«ÊÓƵ

Marta Natalia Wr¨®blewska, a higher education scholar at SWPS University, noted the susceptibility of Poland¡¯s current research assessment system to external interference. ¡°We don¡¯t use metrics like impact factor or journal impact factor but rather a list of journals and publishing houses that is more or less manually compiled by a committee appointed by the minister,¡± she said.

¡°Part of the dissatisfaction with the current system comes from the fact that the list?doesn¡¯t?reflect the actual standing of journals or publishing houses at all.¡±

ÁñÁ«ÊÓƵ

The former minister of education and science, Przemys?aw Czarnek, was ¡°notorious for hand-steering the list¡±, Dr Wr¨®blewska said, with small theological journals receiving similar weighting to some more renowned titles.

Nevertheless, both Dr Wr¨®blewska and Professor Pa?ys said a move towards expert assessment could be hindered by a lack of trust in experts in Poland. ¡°How do you guarantee that the?¡®international peer reviewers¡¯ will be unbiased, objective and resistant to?manipulation? Who will appoint them?¡± asked Dr Wr¨®blewska. The sole focus of expert assessment on high-performing research units, she added, would see weaker units lose out on valuable critical feedback.

Such reforms, both noted, would also come at considerable cost. ¡°[The authors] acknowledge that their proposals make sense only when combined with a significant increase of research funding,¡± Professor Pa?ys said. ¡°The prospects for increasing [government] investment in science are not optimistic at the moment.¡±

Stanis?aw Kistryn, coordinator of the Polish chapter of the Coalition for Advancing Research Assessment and a professor at Jagiellonian University, said the academy proposals were ¡°not good for universities¡±, with the proposed funding structure omitting any financial incentives for teaching excellence.

ÁñÁ«ÊÓƵ

Professor Kistryn also raised concerns about the academy¡¯s plan to fund research units rather than institutions as a whole. ¡°In many universities, faculties are semi-autonomous, but if there is separate funding and separate assessment then we do not have a university any more ¨C we have a federation of faculties, and no common research policy is possible,¡± he said.

Dr Wr¨®blewska suggested that the academy¡¯s proposals for a new assessment system were premature, noting that Poland¡¯s current assessment regulations were established in 2018, with only one round of evaluation, in 2022, carried out to date.

¡°There are details of the regulations that require tweaking for sure, but overhauling the entire system would amount to throwing out the baby with the bathwater,¡± Dr Wr¨®blewska said. ¡°To my mind, what would be helpful is an independent review of the evaluation exercise to assess its influence on the field.¡±

emily.dixon@timeshighereducation.com