OMG academic peer review is totally fucked and it never works and the third reviewer is a stupid idiot asshole!!!11!!!1!
I am certainly guilty of complaining about the review process when my (uniformly awesome) work is the target. Today on twitter I learned that Leonid Kruglyak (@leonidkruglyak) and I have something in common in this regard. He complained:
Obviously, I don't know what the reviewer actually asked for*. And, though I don't often want to admit it, if I'm being honest, most** of the comments from reviewers do end up making the work better. The link in that tweet goes to a column by Hidde Ploegh published a while ago by Nature arguing that the peer review process is a "wasteful tyranny" and that it is worse the higher-up the journal food-chain you are. In this column, he proposes 3 steps to improve peer review:
First, they should insist that reviewers provide a rough estimate of the anticipated extra cost (in real currency) and effort associated with experiments they request. This is not unlike what all researchers are typically asked to provide in grant applications. Second, journals should get academic editors with expertise in the subject to take a hard look at whether the requests of reviewers will affect the authors' conclusions, and whether they can be implemented without undue delay. Third, reviewers should give a simple yes or no vote on the manuscript under scrutiny, barring fatal shortcomings in logic or execution.
I might be mistaken, but it seems to me that these steps would just push all the blame to the editors. I have not seen evidence that the peer review process is much different at places that use academic editors as compared to professional editors.
I think that part of the reason we all complain about peer review is that our papers get rejected. And this isn't going to change any time soon. Most journals, especially the "high-impact" magz, get far more submissions than they can publish. That being said, all of us that complain about peer review are the people we complain about. I am sure that there are times that folks would characterize me as a "third reviewer". I am not an evil asshole, trying to scoop a competitor, or lazy***. My goal is always to write a logic-based, thoughful review that I would be
happy not be pissed off to read as an author. Do I make mistakes? Certainly. But I am trying.
In the end, the only way that peer review will get better is if we "peers" decide to change it. If there are established folks out there (like Drs. Plough and Kruglyak) that act like the "perfect" reviewer, and train the people in their labs to do the same, maybe the cycle of "look, I've read it, I can be as critical as the next dude and ask for something that's not yet in the manuscript" can end.
It will still suck if my paper gets rejected.
*IME, the author's paraphrasing of reviewer requests can be a little hyperbolic.
**I am not arguing that there are exceptions or that EVERY reviewer comment is worthwhile.
***Though these things may happen with some reviewers, I think that they are probably the exceptions, not the rule. Call me an idealistic dim-wit.