Institute for Scientific Information (ISI)

  Scholarly peer review or academic peer review (also known as refereeing) is the process of having a draft version of a researcher's methods and findings reviewed (usually anonymously) by experts (or "peers") in the same field. Peer review is widely used for helping the academic publisher (that is, the editor-in-chief, the editorial board or the program committee) decide whether the work should be accepted, considered acceptable with revisions, or rejected for official publication in an academic journal, a monograph or in the proceedings of an academic conference.

Academic peer review requires a community of experts in a given (and often narrowly defined) academic field, who are qualified and able to perform reasonably impartial review. Impartial review, especially of work in less narrowly defined or inter-disciplinary fields, may be difficult to accomplish, and the significance (good or bad) of an idea may never be widely appreciated among its contemporaries. Peer review is generally considered necessary to academic quality and is used in most major scholarly journals. However, peer review does not prevent publication of invalid research,[1] and as experimentally controlled studies of this process are difficult to arrange, direct evidence that peer review improves the quality of published papers is scarce.[2]

Scholarly peer review has been subject to several criticisms, and various proposals for reforming the system have been suggested over the years. Many studies have emphasized the problems inherent to the process of peer review. (see Squazzoni et al. 2017[3]). Moreover, Ragone et al., (2013)[4] have shown that there is a low correlation between peer review outcomes and the future impact measured by citations. Brezis and Birukou also show that the Peer Review process is not working properly. They underline that the ratings are not robust, e.g., changing reviewers can have a dramatic impact on the review results. Two main elements affect the bias in the peer process.[5]

• The first element is that referees display homophily in their taste and perception of innovative ideas. So reviewers who are developing conventional ideas will tend to give low grades to innovative projects, while reviewers who have developed innovative ideas tend, by homophily, to give higher grades to innovative projects.
• The second element leading to a high variance in the peer review process is that reviewers are not investing the same amount of time to analyze the projects (or equivalently are not with the same abilities). Brezis and Biruku[5] show that this heterogeneity among referees will lead to seriously affect the whole peer review process, and will lead to main arbitrariness in the results of the process.[5]

The peer process is also in use for projects acceptance. (For projects, the acceptance rates are small and are between 1% and 20%, with an average of 10%. In the European H2020 calls, the acceptance rate is 1.8%.) Peer review is more problematic when choosing the projects to be funded since innovative projects are not highly ranked in the existing peer-review process. The peer-review process leads to conformity, i.e., the selection of less controversial projects and papers. This may even influence the type of proposals scholars will propose, since scholars need to find financing for their research as discussed by Martin, 1997:[6] "A common informal view is that it is easier to obtain funds for conventional projects. Those who are eager to get funding are not likely to propose radical or unorthodox projects. Since you don't know who the referees are going to be, it is best to assume that they are middle-of-the-road. Therefore, the middle-of-the-road application is safer".[5]

Other attempts to reform the peer review process originate among others from the fields of metascience and journalology. Reformers seek to increase the reliability and efficiency of the peer review process and to provide it with a scientific foundation.[7][8][9] Alternatives to common peer review practices have been put to the test,[10][11] in particular open peer review, where the comments are visible to readers, generally with the identities of the peer reviewers disclosed as well, e.g., F1000, eLife, BMJ, and BioMed Central.[12] In the case of eLife, peer review is used not for deciding whether to publish an article, but for assessing its importance and reliability.

References

1.  Kupferschmidt, Kai (August 14, 2018). "Researcher at the center of an epic fraud remains an enigma to those who exposed him". Science | AAAS. Retrieved August 11, 2019.
2.  Couzin-Frankel J (September 2013). "Biomedical publishing. Secretive and subjective, peer review proves resistant to study". Science. 341 (6152): 1331. doi:10.1126/science.341.6152.1331. PMID 24052283.
3.  Squazzoni, Flaminio; Brezis, Elise; Marušić, Ana (October 1, 2017). "Scientometrics of peer review". Scientometrics. 113 (1): 501–502. doi:10.1007/s11192-017-2518-4. ISSN 1588-2861. PMC 5629222. PMID 29056787.
4.  Ragone, Azzurra; Mirylenka, Katsiaryna; Casati, Fabio; Marchese, Maurizio (November 1, 2013). "On peer review in computer science: analysis of its effectiveness and suggestions for improvement". Scientometrics. 97 (2): 317–356. doi:10.1007/s11192-013-1002-z. ISSN 0138-9130. S2CID 16803499.
5.  Brezis, Elise S.; Birukou, Aliaksandr (April 1, 2020). "Arbitrariness in the peer review process". Scientometrics. 123 (1): 393–411. doi:10.1007/s11192-020-03348-1. ISSN 1588-2861. S2CID 211017926. CC BY icon.svg Text was copied from this source, which is available under a Creative Commons Attribution 4.0 International License.
6.  Martin, B. "Chapter 5: Peer review as scholarly conformity". www.bmartin.cc.
7.  Rennie, Drummond (July 7, 2016). "Let's make peer review scientific". Nature News. 535 (7610): 31–33. Bibcode:2016Natur.535...31R. doi:10.1038/535031a. PMID 27383970. S2CID 4408375.
8.  Slavov, Nikolai (November 11, 2015). "Making the most of peer review". eLife. 4: e12708. doi:10.7554/eLife.12708. ISSN 2050-084X. PMC 4641509. PMID 26559758.
9.  Couzin-Frankel, Jennifer (September 18, 2018). "'Journalologists' use scientific methods to study academic publishing. Is their work improving science?". Science | AAAS. Retrieved July 18, 2019.
10.  Cosgrove, Andrew; Cheifet, Barbara (November 27, 2018). "Transparent peer review trial: the results". Genome Biology. 19 (1): 206. doi:10.1186/s13059-018-1584-0. ISSN 1474-760X. PMC 6260718. PMID 30482224.
11.  Patterson, Mark; Schekman, Randy (June 26, 2018). "A new twist on peer review". eLife. 7: e36545. doi:10.7554/eLife.36545. ISSN 2050-084X. PMC 6019064. PMID 29944117.
12.  Ross-Hellauer, Tony (August 31, 2017). "What is open peer review? A systematic review". F1000Research. F1000 Research Ltd. 6: 588. doi:10.12688/f1000research.11369.2. ISSN 2046-1402. PMC 5437951. PMID 28580134.

Edit This Article