Quiz on public peer review

I have been required to complete a “responsible conduct of research” training module by the research office at my school. The reason I am commenting is that I was asked to answer the following question “true” or “false”. This is not meant to be an opinion, it is from a quiz on which I am supposed to maximize my score.

True or false: “A good alternative to the current peer review process would be web logs (BLOGS) where papers would be posted and reviewed by those who have an interest in the work.”

I thought that would be interesting to people who follow Boole’s rings. The correct answer, of course, was “false”. Here is the rationale they gave:

“Although the peer review process is evolving, the described system would probably not work very well. It is likely that the peer review process will evolve to minimize bias and conflicts of interest. It is, in the best interest of everyone involved in the research enterprise that the scientific review process be fair and rigorous.”

Just throwing it out there….

This entry was posted in Musings. Bookmark the permalink.

8 Responses to Quiz on public peer review

  1. Peter says:

    This would be hilarious if it wasn’t real.

    The answer is “it won’t work” and then some general bs about peer review evolving (how exactly?) to become awesomesauce!!eleventy!! stuff — how, uh, refined.

    At least the reverse implication of the predicted evolution is honest: right now, peer review is biased, full of conflicts of interest, neither fair nor rigorous. Now that’s something we could agree on…

    • Carl Mummert says:

      I’m a moderate about peer review – it certainly has some benefits when done well, and problems when done poorly. But the idea that peer review will magically evolve on its own to eliminate problems seems to be particularly farfetched.

      • Peter says:

        I exaggerated a little. I also believe in peer review in principle. But the inflation of papers makes “when done well” a more and more remote possibility.

  2. “True or false: [insert idea we don’t like]”


    Conform or be punished.

  3. Victoria Gitman says:

    Well, I always suspected that the BR crowd were troublemakers, but now it turns out they are aspiring “irresponsible” researchers as well! On a more serious note, the presence of such a question does seem to indicate a certain level of insecurity in the establishment.

  4. Peer review is the only service still provided by the scientific publishing industry. They have to cling on this last thread, like for example in the case of “gold open access” movement, where they are basically charging authors for the peer review process while offering the dubious reward of not putting a paywall in front of their papers.

  5. Matthew B says:

    I think there is no doubt the peer-review process is corrupted: I’m sure I am not the only one who has seen reviewers with motives/agendas reject work worthy of being published. I’ve always thought the problem WAS peer-review, so why not just abolish it? Personally, I think ArXiv is the solution to this problem. It is free, there is no copyright, and people can make up their own minds as to whether or not they think it is good.

    • Carl Mummert says:

      Thanks for the comment. I do want to point out that I have the copyright to all my papers on the arXiv. I don’t know whether any papers there are in the public domain, but the default license grants the arXiv just enough rights to publish the paper, while reserving all other rights for the author. The arXiv, for mathematics, does resolve many issues with peer review, from the delay in dissemination to the cost of paywalls. This, and the practice of speaking about results immediately, long before formal publication, makes the peer review system in mathematics essentially a secondary mechanism for establishing credentials, rather than the primary mechanism for publishing. Unfortunately, there are many scientific disciplines that have nothing analogous to the arXiv, and which do not tend to publicize results before formal publication. The situation in those areas is very different.

Leave a Reply

Your email address will not be published. Required fields are marked *