ÁñÁ«ÊÓƵ

Peer reviewing is becoming more cavalier, self-serving and ignorant

<ÁñÁ«ÊÓƵ class="standfirst">The professionalism and collegiality with which editors used to respond to complaints has all but disappeared, says Harvey Graff 
June 2, 2022
A peer reviewer with monocle
Source: iStock

Recently, three months after submitting an article to a scholarly journal, I received an unusually short rejection. Referring to one review, it simply read: ¡°In view of the criticisms [none mentioned]...we cannot publish your manuscript.¡±

When I wrote back asking about the number of reviews and their contents, I received two reviews. These were also surprisingly brief, as well as unknowledgeable, confused and self-contradictory. No examples accompanied any of their sweeping condemnations. No suggestions were offered. One referred to unidentified literature and ¡°experts¡± that do not exist.

In more than 50 years of publishing in professional journals, as well as non-specialist outlets, I had never seen anything like this. I asked to speak to the editor, something I had done a number of times previously. The editor responded by patronising and then demeaning me. ¡°I think the essay would do great as part of a book or a series of expert opinion pieces. But it is very difficult to write very long journal papers that have multiple research questions without establishing a very clear analytical method.¡± Not even a hint of what that might be.

Most unprofessionally, the editor resorted to victim-blaming. ¡°Perhaps [your] long publication history is the problem. People get so used to publishing; they can¡¯t imagine criticism and blame the reviewers. It¡¯s okay to get a rejection; it¡¯s how we learn.¡± This to an author of hundreds of articles and several dozen books, whose questioning of editors, with examples and evidence, has often resulted in serious exchanges and sometimes further review.

ÁñÁ«ÊÓƵ

ADVERTISEMENT

Not this time. Now I was accused of making the exchange ¡°acrimonious¡±.

The intellectual, professional, collegial form of respect that editors used to display in response to complaints about reviews seems to have all but disappeared. Relatedly, editors seem to be less experienced, less committed and involved, and less knowledgeable than they used to be. For instance, they only occasionally respond to specific questions about topics or form/format prior to formal submission. Many categorically refuse.

ÁñÁ«ÊÓƵ

ADVERTISEMENT

There has also been a decline in the clarity and consistency of journals¡¯ instructions. I recently submitted an article to a journal published by a major professional organisation in response to a call for papers for a special issue. The regular editor and one of the two guest editors for the issue said they ¡°wanted it.¡± Shortly thereafter, a further set of authors¡¯ instructions was circulated, with a word limit shorter than my draft and a specific reference style sheet. I revised to meet the new specifications.

Almost four months went by. Then, nearly two weeks after the announced date for final decisions, I received a message from the second, previously silent guest editor. Because of the number of submissions, they had now substantially cut the word limit and changed the guidance for references. This came with ¡°suggestions for revision¡± that made no sense. An editor at the journal¡¯s main office generously stepped in and assisted me. But I have lingering confusion and questions about the role and conduct of guest editors, one of whom was also the editor of another journal.

To be fair, some respectful editors (although, more often, editorial assistants) have been willing to submit my manuscripts for me when I have been unable to fathom their journal¡¯s online submission systems ¨C or those systems don¡¯t work at all. But they need to do more to overcome the iniquities of these ubiquitous but always slightly different systems.

Such systems may ease the tasks of moving manuscripts from author to editor to reviewers and back again, but their format seems to me to encourage superficial reviews. One editor recently responded to my complaints by explaining that the reviewer ¡°prefers to write, briefly, as if she had to write longer, her view would come across as even more negative¡±. But if she had used the extra space to support her criticisms with actual examples, the opposite would have been true.

ÁñÁ«ÊÓƵ

ADVERTISEMENT

In part, this worsening problem also stems from the fact that reviewing journal submissions and even books is hard to classify, evaluate and plug into the messy calculations of annual reviews, tenure, and promotion. Beyond that, graduate students and new professors are rarely taught how to review. I was unusually fortunate in that my graduate adviser supervised a then state-of-the-art urban social history project. Our group of students, staff and faculty, with visitors from other universities, met regularly and critiqued each other¡¯s thesis chapters, article drafts and our adviser¡¯s working papers. Why does that remain so uncommon?

My conclusions are shared by others with whom I have consulted, including past and present editors, editorial board members, reviewers, and authors. A younger colleague with an unusual amount of editorial experience tells me, ¡°Reviewers don¡¯t seem to understand the purpose of peer reviews and especially don¡¯t know how to give constructive feedback.¡± They see more ¡°terrible reviews¡± including ad hominem attacks from the reviewers for a journal for history undergraduates.

There was never a ¡°golden age¡± of journals. Reviewing in the humanities and social sciences was always hit-and-miss. But if it really is becoming more cavalier, more self-serving and promoting, and less characterised by close reading and relevant expertise, we clearly have a huge problem. Fair hiring, promotion and grant distribution depend on everyone doing their bit to uphold the highest possible standards.

Harvey J. Graff is professor emeritus of English and History and Ohio Eminent Scholar at The Ohio State University. His Searching for Literacy: The Social and Intellectual Origins of Literacy Studies is forthcoming this summer.

ÁñÁ«ÊÓƵ

ADVERTISEMENT

Register to continue

Why register?

  • Registration is free and only takes a moment
  • Once registered, you can read 3 articles a month
  • Sign up for our newsletter
Register
Please Login or Register to read this article.
<ÁñÁ«ÊÓƵ class="pane-title"> Related articles
<ÁñÁ«ÊÓƵ class="pane-title"> Reader's comments (6)
In todays world, journals should be redundant. Clearly it is not for dissemination of the work. There are a myriad of ways by which you can do that and for zero cost in many cases. Stop sending papers to journals and let us start with the seniors, who might be sitting on tenure committes.
I am disheartened by this opinion piece which makes sweeping generalizations with no evidence other than one personal bad experience. I¡¯m sorry to hear this eminent professor - or any other scholar- had this experience. But jumping from one experience to these statements is less than scholarly: ¡°The intellectual, professional, collegial form of respect that editors used to display in response to complaints about reviews seems to have all but disappeared. Relatedly, editors seem to be less experienced, less committed and involved, and less knowledgeable than they used to be.¡± The article moves on to complain about confusing comments from an unknown guest editor - fair criticism- to complaints about online submission systems. As a reader, I am confused -is the problem the reviewers, the editors, the online submission systems? The frustration is clear; the line of argument is not. I would hope for a more substantial opinion piece on this important topic.
Maybe my experience is a bit different because I work in a different area, in Mathematics. Anyway, nowadays the number of submissions has blown up but the number of really expert reviewers is quite limited. Thus, reviewers are asked to review more and more papers and consequently their work becomes less and less accurate.
Sorry Commentor 2; reread the essay. "But jumping from one experience to these statements is less than scholarly" I refer to the experience of many colleagues and former doctoral students, and quote at length a younger colleague who has extensive editorial experience. Criticize away but not don't parallel the reviewers who I and MANY others find unprofessional. In less than 1000 words, just how much evidence do you expect? Now tell all of us how you define "less than scholarly"?
Peer reviewing is becoming more cavalier, self-serving and ignorant because the academic cabals that perpetuate the system are becoming more cavalier, self-serving and ignorant.
I publish in various fields of applied economics where mathematical models are frequently used. One major problem is that often papers are published and proposed using these models but where no detailed documentation of the models is provided. Thus, there is no way a reviewer in the field who is knowledgeable in general about such models can do an adequate peer review of any paper which relies on one or more undocumented models. Yet, journals routinely publish papers based on such models. There are probably hundreds of papers like this published each year in applied economics. To me, the scholars who must have peer reviewed and approved such papers for publication are being totally irresponsible, as are the editors who let these peer reviewers be determinative for publication. All such editors should be fired for unethical behavior in doing or promoting science. In addition, I have had many peer reviews of my papers where the reviewer clearly did not understand my arguments. Yet, journal editors now do not seem to know enough themselves to understand when this happens when I point it out to them. They assume that any peer reviewer that they have chosen must be sufficient knowledgeable to review any such paper in that field. This is a poor assumption, especially since most peer reviewers do not seem to put enough time into performing their reviews.
<ÁñÁ«ÊÓƵ class="pane-title"> Sponsored
<ÁñÁ«ÊÓƵ class="pane-title"> Featured jobs
ADVERTISEMENT