Articles

Pushing the limits of scientists

The life of an aspiring scientist is grueling: long hours in the lab, little sleep, and the rest of the time is spent desperately trying to publish results. Even when young scientists formulate a solid theory, supported by valid experiments, journals find their work inadequate. What more do they want? They want scientists to conduct tangential follow-up experiments. Well, now one scientist is fighting back. Biologist Hidde Ploegh (Whitehead Institute of the Massachusetts Institute of Technology) objects that the increased burden journals put on scientists hinders the peer review process.

At first, it may seem that journals wanting scientists to conduct more experiments is good: one can never be too sure. Ploegh’s concern, though, is that the requested experiments tend to be tangential—ultimately irrelevant to the original conclusions of the submitted paper. Journal editors frequently take the liberty to demand and design new experiments of their prospective authors, refusing to publish the piece otherwise.

Journal commanded experiments have been dubbed “reviewer experiments.” The more prestigious the journal, the more likely the journal will request reviewer experiments. The already time-constrained scientist must choose between spending time on these new, largely tangential reviewer experiments, or finding a different (less prestigious) journal that may accept the research as is. Prestigious journals feel obliged to uphold their outstanding reputation, and so can justify a reviewer experiment for practically every submission. In the end, such journals publish higher quality papers (higher because of the additional reviewer experiment) without any additional cost for them.

Not content with only criticizing the peer review system, Ploegh offers three suggestions to help fix it. First, journal editors who ask for reviewer experiments should estimate the cost of their proposed experiments. “Cost” includes not only a hard dollar amount but also time and effort. Second, before requesting reviewer experiments, reviewers should consult experts who can tell them whether or not the reviewer experiment will actually impact the author’s original conclusion in any significant way. In other words, reviewer experiments should not be tangential. If the expert deems the reviewer experiment as relevant, then he or she should also note how quickly such an experiment can be performed. Third and finally, Ploegh advocates for journals to simply accept or reject submissions as is. If the editors find errors in the research, or believe the research lacks originality, then, and only then, should they reject it. On the other hand, if the submission demonstrates solid methodology and originality, it should be published.

Ploegh is confident that with such changes in place, the peer review process will function more smoothly, save scientists’ time, and encourage editors to provide constructive feedback for authors, rather than piling on work that may or may not help the authors’ original argument.

While all of Ploegh’s suggestions are reasonable, it seems unlikely that they will fully address the problem. The problem, at its core, is the fact that science has a social aspect to it. No matter how tight one tries to write the rules, the social factor of science (and of the peer review process in general) inevitably comes into play. For example, imagine that a respected scientific journal receives three times as many papers as it will publish. Suppose further that a third of these fail Ploegh’s criteria; the journal still needs to weed out half of the remaining papers. All of them are rigorously argued and original, but half will be rejected. How will the journal decide which ones to publish? Probably through subjective, social “reasoning.” The big name authors will certainly be published, and after that who knows what aesthetic sensibilities an editor will use? The peer review process certainly will be improved by Ploegh’s suggestions, but can it ever be truly fixed given the human element?

For more, see “End the wasteful tyranny of reviewer experiments” in Nature News.