Eugenio Picano was right when, years ago, he talked about Ulysses’ syndrome in relation to scientific papers: any paper, whether good or bad, helpful or useless, sooner or later finds a port to land on [1]; something else all together than failure. The issue is how long it takes to land somewhere, amidst anthropophagus giants, stunning nymphs and seducing mermaids…
The infinite wait
From the researchers’ point of view the period of time between the paper submission and the first response from the publishing journal is crucial: an immediate rejection (whether due to formatting or content issues or because the work is not considered appropriate for the journal) is somehow tolerable; it is definitively preferable to waiting a very long time for the first review round [2]. The duration of the peer review process depends on the field of the paper: it is faster for chemistry or engineering and longer for economy and business management. On average, an article submitted to a medical journal takes little more than four months to go through the entire peer review process (whether the outcome is positive or negative) and, once approved, between four to five months to get published. These excessively long timelines are common for almost every journal, although the most well known ones- thus with a larger workgroup and more organised procedures- take a slightly shorter time to reach a decision and to publish the paper [3].
In order to make the process faster some of the journals from the Springer Nature editorial group adopted a “portable” peer-review revision policy: the journals are open to consider papers based on the comments received from the responsible reviewers of other journals, including those that are not printed by other publishers. This makes it easier to share work between journals, publishing the reviews and the referees’ identity. The C19 Rapid Review initiative also seems interesting: it is a collaboration between editors and institutions that requires reviewers to commit to the provision of quick reviews and, whenever the papers are sent to different journals, to agree to their identities being shared with the directors of the different journals.
The covid-19 pandemic forced many publishers to accelerate the change of their procedures. The “paper pandemics”, indeed, impacted both the most and less known journals [4], and in several cases it was inevitable to explore solutions to promptly review the articles received, regardless of the size or the prestige. Journals and editors aim to attract referees that can help publishing quickly new information that is useful to face the healthcare emergency. The Medical Journal of Australia created preferential routes for covid-19 related research with rapid reviews and pre-print spaces. The journal management pre-emptively said that making everything faster can certainly cause mistakes, but delaying publication can have even worse consequences [5].
The Royal Society of Open Publishing also created a priority route for covid-19 related content, with a group of 700 referees that committed to reviewing papers in 24-48 hours from the moment they take them on [6]. In the journals of the American Medical Association (the JAMA Network) the editorial staff took over the peer review process [7]. Instead the journal eLife decided to simplify for authors by protecting them from any eventual misleading request that comes from excessively strict referees [8]. Basically, if the paper is related to covid-19 there seems to be a lower risk of failure; however, it should be considered that the short lifespan of content is a great cause of frustration among authors: if what you wrote is not published quickly, the risk that it might become dated content is very high.
On the other hand, something very similar takes place in research, where, between the submission of a clinical trial proposal and its approval (or even just the response from the institutions) the waiting time is always longer than reasonable. This leads to proposals that, once approved, are less up-to-date (or motivated).
The infinite arrogance
However, the rapidity of the assessment should not be confused with rushing it or being sloppy. An aspect that is emerging in recent years- and that obviously does not have anything to do with the pandemic- is that increasingly more often the referees’ opinion has a mocking tone or contains statements that are offensive towards the authors. As Jeff Clements, a leading expert in marine ecology, disclosed: “A study I did with a few colleagues in my field- ecology and evolution- demonstrated that 10-35 per cent of peer reviews contains disrespectful passages and 43 per cent of the reviews features at least one comment that is inappropriate for an exchange of opinions among professionals. Personally I had to withstand comments on the line of ‘I would not even define the authors’ work as science’” [9]. In such cases the feeling of frustration doubles, because while the reviewer’s constructive comments can contribute substantially to the quality of the scientific papers, quite often it is impossible to respond to superficial, contradicting or inappropriate comments.
I love journals that make you complete a mountain of paperwork before they desk reject a paper. Vinay Prasad [Twitter, August 11, 2020]
Something that tends to be overlooked is that directors are faced with an increased need for collaborators to achieve peer reviews due to the multiplication of scientific journals. The vast majority of researches, though, does not know how to do this because they never received adequate training. On top of that, it is even more rare that they get taught about the etiquette of peer review [10]. Even though being educated about courtesy after a certain age is very challenging, the ability to critically appraise literature should be an essential skill for healthcare professionals and being able to read between the lines of a piece of research should be common practice for any specialisation training.
When discussing the frustration of researchers that stems from being under a form of abuse of power we should not forget to also mention their discomfort- a euphemism really- towards the fact that despite being the ones who came up with the idea of the study and who conducted it they see their name sandwiched between the names of the department director and their direct manager. It is a sandwich that is hard to digest, especially if one is a bit more inexperienced.
The infinite cloudiness
Being sufficiently competent to assess the methods and the results of a study requires an effort. It is much easier not to be very kind, especially whenever the peer review is single blinded, so to say: a referee masked by anonymity not only can afford to be insolent but can also take revenge on his colleagues. In a survey by the journal Nature [11], that collected the responses of 1,230 referees, 82 per cent of the participants agreed that the traditional peer review process effectively guarantees good quality work getting published. Nevertheless, 63 per cent of the interviewed participants suggested alternative refereeing methods to the directors of the journals. More than half of the people that responded to the survey also demanded for more transparency.
Speaking of transparency, some journals give referees the option to choose between remaining anonymous or being listed as a referee to the work they assessed. In the last three years roughly 3,700 referees in the natural sciences field decided to be publicly acknowledged for their efforts and roughly 80 per cent of the papers published on Nature listed at least one referee. 55 per cent of referees chose to be listed; 26 per cent refused and 19 per cent did not respond at all [11].
Still speaking of transparency- and the peer review method- the decision of the Lancet journals group to add a paper submission requirement, whereby access to the raw data of the study up for review has to be guaranteed for more than one author, was recently in the news [12].
Long wait, arrogance and cloudiness: what to do?
After years of global conversation the traditional peer review process might not be appropriate any longer [13]. For a research that intends to meet the citizens’ health needs promptly, interminable waiting times, frustrations and lack of transparency are an unsustainable toll to pay. Furthermore, an in-depth assessment of the methods and results of a study can only take place when having access to raw data and it is not conceivable that a referee- whether voluntary or paid- can delve into this type of analysis and appraisal. It would be better if the research data were freely accessible on open and public data banks. Academic journals would then take on the crucial task of creating a space for and facilitating exchange of opinions and discussion. In a similar scenario the referees’ task would certainly be less cumbersome and authors might be more accepting of their assessment.
A young oncologist, Bishal Gyawali, recently disclosed on Twitter that he responded to the request to review a specialist article in the following manner: “I was happy I read it: this paper should be approved”. Apparently the directors of the journal did not appreciate this response very much, as they expected their checklists and valuation grids to be filled out. However, it is very likely that Bishal was right and that a true “peer-review” should work this way.
References
[1] Picano E. la dura vita del beato porco. Seconda edizione. Roma: Il Pensiero Scientifico Editore, 2007.
[2] Huisman J, Smits J. Duration and quality of the peer review process: the author’s perspective. Scientometrics 2017;113:633-50.
[3] Björk BC, Solomon D. The publishing delay in scholarly peer-reviewed journals. J Informetrics 2013;7:914-23.
[4] Dinis-Oliveira RJ. Covid-19 research: pandemic versus “paperdemic”, integrity, values and risks of the “speed science”. Forensic Sciences Res 2020;9:1-4.
[5] Talley NJ. Sars-cov-2, the medical profession, ventilator beds and mortality predictions: personal reflections from an Australian clinician. Med J Australia 2020;212:1.
[6] Brock J. Rapid Registered Reports initiative aims to stop coronavirus researchers following false leads. Nature Index, 14 aprile 2020.
[7] Golub RM. Editorial evaluation and peer review during a pandemic: how journals maintain standards. JAMA 2020;324:453-4.
[8] Eisen MB, Akhmanova A, Behrens TE, Weigel D. Peer review: publishing in the time of covid-19. eLife 2020;9:e57162.
[9] Clements JC. Don’t be a prig in peer review. Nature 2020;585:472.
[10] Nicholas D, Watkinson A, Jamali HR, et al. Peer review: still king in the digital age. Learned Publishing 2015,28:15-21.
[11] Walton R. Recognising the contribution of Nature Research journal referees. Of schemes and memes Blog, 16 aprile 2020.
[12] Hopkins JS. Lancet medical journal changes peer-review process amid flurry of covid-19 research. Wall Street Journal, 18 settembre 2020.
[13] Addis A, De Fiore L. La disseminazione della ricerca. Recenti Prog Med 2019; Suppl. Forward; Maggio.
© 