- Open Access
Q&A: Re-review opt-out and painless publishing
BMC Biology volume 11, Article number: 18 (2013)
What exactly is re-review opt-out - briefly?
It's an editorial policy. Any author whose research paper is judged by referees to bepublishable in BMC Biology subject to important revisions - which may mean thecollection of additional experimental data - will be asked to choose whether he or shewishes the paper to be seen again by the reviewers after revision.
What motivated the policy?
Starting with a letter to Science in 2008 , there has been a succession of complaints about the frustrationsand in some cases quite serious problems engendered by the increasingly protracted anditerative reviewing practices of, in particular, the higher-profile journals (forexample, [5–7]). One of our Editorial Board members suffered a particularlyegregious example and appealed to us for help (his story will be published shortly as apodcast), and the policy was launched as a direct consequence.
We (at that time, Journal of Biology - the fusion with BMC Biology came after the policy was established) decided to operate, for an experimentalperiod, a policy whereby in the case of important but non-lethal criticisms by referees,authors would be asked to address the issues raised, and then to choose whether thereferees saw the paper again or not.
Don't you run the risk of publishing invalid papers?
Any journal does that, under any system of review: no system is perfect. The question iswhether ultimately it is the editors' responsibility to ensure the soundness of thepublished paper, or the authors'. Re-review opt-out places the onus on the authors. Isthat likely to result in the publication of more invalid papers than under othersystems? We think not.
Really? Surely insistence on satisfying referees must reduce the risk of publishinginvalid papers?
In principle, yes. In practice, first of all, when authors opt out of re-review theirrevised manuscripts are scrutinized by the editorial team, and may be rejected if itseems clear that the problems identified by the referees have not been addressed.Second, don't forget, authors can opt in to re-review - and about half of them do.(Sometimes an author will say he/she would prefer the referees to see the paper again,but wants to avoid the delay to publication, and in those cases we ask the referees ifthey will return reports within a week, and proceed without their input if theydon't.)
Third, iterative re-review is exhausting for referees, and after two or three iterationswhere authors have revised inadequately, the referees can simply lose patience andrecommend publication. We think it unlikely that significantly more inadequatelysupported papers will be published under re-review opt-out than under the more usualsystem; and it will have the substantial advantage of lessening the burden on a limitedpool of referees.
What about the referees? Surely they are likely to be unwilling to assess papers ifthey think their criticisms may be ignored?
We were afraid of this. We do state of course in our invitation to referees that authorswill be offered a choice about whether their revisions are seen by their reviewers. Inpractice, we have almost never had a referee refuse to assess a paper for thatreason.
And it is not the case that their criticisms may be ignored: as already stated, if theauthors opt out of re-review, the editors look carefully to see if important criticismsseem to have been addressed, and the paper may be rejected if the response seemsinadequate - for example, if the authors have argued that additional data requested bythe referees are unnecessary, or if it seems that additional evidence provided is notpersuasive.
If the editors decide to reject the paper, can the authors change their mind andappeal to the referees?
Yes. This occasionally happens, and in those cases the referees sometimes advise onbalance that the paper should be published. It is not always straightforward to decidehow much evidence should be required.
Do you allow authors to choose to go back to some referees but not all of them?
No. It is not uncommon for (say) two out of three reviewers to make positiverecommendations, while a third has criticisms that seem to need to be addressed, andauthors do sometimes in those cases argue that the third referee is mistaken and shouldbe disregarded.
In a case like that, we consult all three referees, but we send each referee all thecomments on the original version of the paper, along with the authors' responses, andask the favorable referees to comment on the criticisms of the unfavorable one. We mayalso seek adjudication from a fourth reviewer or an Editorial Board member, if theauthors request it, or there is no consensus among the existing referees.
You are a general biology journal with a policy of selecting papers of some generalinterest or importance - is it part of the referee's job to evaluate that?
No. All submissions to BMC Biology are screened first for suitability ongrounds of their interest or importance in principle for the journal - usually by anEditorial Board member, sometimes (if there is no appropriate Editorial Board member, ornone is available) by an appropriate expert not on our Board. Only if the paper isjudged suitable on this first screen is it sent to referees, and the referees are thenasked to evaluate the paper solely on the basis of its technical soundness and not onthe basis of its interest.
But surely referees may be more expert, or may look at the paper more closely, thanyour Editorial Board members, and have a different view of its claims to specialinterest or importance?
Usually, the Editorial Board member is judging the paper's claim at face value and onthe basis of his or her expert general perspective rather than examining the paper indetail, and giving an opinion on that basis. Of course it is just an opinion, andopinions vary - it is in the nature of a selective journal that it must draw lines on acontinuum from highly specialized to extremely interesting and/or important to allbiologists. In the absence of expert consensus, it is the editors' decision where theline should be drawn. BMC Biology is usually generous.
Because it is not the Editorial Board member's job, however, to evaluate the technicalsoundness of submissions, it is not uncommon for referees to report that although theclaim of a paper is interesting, it is overstated in the light of detailed scrutiny ofthe data. In such a case, the paper may contain perfectly valid data supporting a pointof relatively specialized interest, but inadequate data in support of its more strikingconclusions.
So in those cases, you reject the paper?
Yes and no. Usually, we offer authors the option of extending their paper to bolster itsmore interesting claim, or resubmitting to one of our subject-specific sister journals,which (in the case that the paper is otherwise completely sound) may simply publish iton the basis of the existing referees' advice. Depending upon how much work we thinkwould be involved in supporting the authors' more striking claim, we may either rejectthe paper but encourage resubmission to a sister journal (with the option ofresubmission to us if they think they can provide the necessary data), or we may inviterevision for BMC Biology but with the proviso that the revised paper may seemmore appropriate for a sister journal.
If the paper is resubmitted to a subject-specific sister journal in the BioMedCentral series, will fresh referees be consulted?
Normally a decision can be made on the basis of the reports sent to BMCBiology. The entire file on the paper can be made available to the sister journal,so that in some cases the paper can simply be accepted as it stands in a sister journal;in others, authors may need to revise the paper to meet criticisms other than thosebearing on the disputed point of general interest.
Isn't the issue of how well a claim needs to be supported also a judgment call?
Yes. We have already acknowledged this above. In straightforward cases where a paper isreporting a significant step in an established field with well validated technology, itis likely to be relatively simple to say how well the conclusions should be supported bythe data.
It is not so easy in fields where the data are by their nature fragmentary or hard tointerpret (phylogenetic data come to mind, whether fossil or genomic); and it is alsoless easy where new ground is being broken (see the comment on the discovery of cyclinsin ). If we turned away all papers whoseconclusions were not solidly grounded in impeccable data, we should risk becoming a verydull journal. Part of our aim is to provide a platform for papers that may becontentious for one reason or another: they make people think.
Are you saying all impeccable papers are dull?
Certainly not. Only that a paper doesn't have to be impeccable to be important, and weshouldn't expect papers breaking new ground to have every i dotted and t crossed (to mixmetaphors).
So are you saying you publish papers that may be wrong just to be provocative?
No. It would be irresponsible to publish flimsy papers just to provoke. Papers that makeclaims that are open to question, or provocative, should be well supported within thebounds of what is possible or reasonable, or should at least make it clear whereevidence ends and speculation begins.
But surely you are misleading your nonspecialist readers if you publish suchquestionable papers?
That is a risk. So when we do publish papers that require some qualification notprovided in the paper itself, we commission expert commentary to put the paper inperspective.
So the authors can say what they like about their data, no matter how outrageous?
No. We do think that what is in a paper is ultimately the responsibility of the author.But it is the responsibility of the editor to decide (on the basis of expert advice fromreferees) what should be published. We may err on the side of publishing an author'sinterpretation even if not fully endorsed by referees, provided that the authors clearlystate that it is what they propose, and not that it is compelled by the evidence. But weshould not publish a paper making a claim that is flatly wrong - either on the basis ofevidence in the paper, or on the basis of others' evidence. Nor should we publishmisrepresentation of others' work. In such cases, publication - provided that the paperis otherwise sound and interesting enough for BMC Biology - would be contingenton the authors' modifying their text appropriately.
Do you think you have answered all the questions arising from this policy?
Almost certainly not: please send us yours.
Robertson M: What are journals for?. J Biol. 2009, 8: 1-10.1186/jbiol111.
Robertson M: Plus ça change. BMC Biol. 2010, 8: 44-10.1186/1741-7007-8-44.
Robertson M: Pit-bull reviewing, the pursuit of perfection and the victims of success. BMC Biol. 2011, 9: 84-10.1186/1741-7007-9-84.
Raff M, Johnson AD, Walter P: Painful publishing. Science. 2008, 321: 36-
Ploegh H: End the wasteful tyranny of reviewer experiments. Nature. 2011, 472: 391-10.1038/472391a.
Petsko GA: The one new journal we might actually need. Genome Biol. 2011, 12: 129-10.1186/gb-2011-12-9-129.
Vosshall LB: The glacial pace of scientific publishing: Why it hurts everyone and what we cando to fix it. FASEB J. 2012, 26: 3589-3593. 10.1096/fj.12-0901ufm.