Publish or perish: Is the peer review process fit for purpose?

Francesca Soutter, BBSRC policy intern at the Society of Biology and PhD student at the RVC, has been following the often heated discussions around the future of publishing.

The peer review process is often considered as a pillar-stone of excellent science. However, the process first introduced in the 1700s has come under increasing scrutiny with the retraction of a number of high-profile scientific papers. There are concerns that the process is not transparent, can be biased and crucially fails to identify serious flaws and fraudulent data.  This is one part of research culture that is being studied by the Nuffield Council on Bioethics Research Project.

Typically editors assess the paper for relevance, novelty and quality, after which the paper is rejected or sent out for peer review. However, the definition of ‘novelty’ can be contentious and difficult to assess, particularly with the number of new journal articles published each week. Recently some journals such as PLoS One have rejected the novelty concept and will publish as long as the research carried out is robust.

If the article makes it past the editor it will be sent for peer review, most often to two or more reviewers selected by the editors. Often journals allow authors to nominate reviewers; in most cases this is appropriate, however, conflicts of interest can arise, for example where previous collaborators are nominated or when fraudulent contact details are given. There is also potential for bias on the part of the reviewer, since reviewers are anonymous but authors are not. This can be an issue in competitive fields.

Is the peer review process able to identify incorrect and even fraudulent papers?  Typically reviewers scrutinise papers, assessing the scientific method, data and conclusions.  However, it is often impossible to detect erroneous papers since it is not possible to test for reproducibility during the peer review process. There is reliance on the integrity of submitting authors for the peer review to work but these same researchers need to publish for career progression.  This conflict of interest can result in publication of poor quality studies that are rarely repeated before publication with overstated results.  Necessity for a strong publication record can also open the door for unscrupulous researchers who seek to publish fraudulent data at best and fraudulent results at worst.

Following publication, concerned readers can contact the journal editors who will then decide whether to retract the paper. This process can take a significant amount of time, in which the paper has often been in the public domain or at very least accessible to the scientific community. This is of particular concern for highly cited papers that can exert significant influence on the field within a short period of time.

Changes to pre-publication peer review aimed to make the process more rigorous and/or transparent are already taking place within some journals. These include anonymised reviewing for both reviewer and author, expertise employed to further scrutinise statistical method and open reviewer comments for retracted articles.

Post-publication peer review sites such as PubPeer are also becoming more common and allow peers to comment on published articles and authors to respond. Whilst this system might facilitate increased scrutiny of scientific findings and potentially reduce fraud, it is open to abuse, giving competitors with an axe to grind access to an often anonymised forum.

So is the traditional peer review system allowing poor science to slip through the net? It is difficult to know how widespread problems within the peer review system are but it is clear that when the system fails it is often highly publicised and controversial. Perhaps we must look at researcher behaviour and culture and encourage discussion and even self-retraction when necessary. However, in the scientific world, where citations are king and researchers must ‘publish or perish’ the turning of the tide may still be some way off.