Friday, January 21, 2011

Peer Review... Reviewed

This is a tad askew from the usual subject matter of this blog, but yet related... "Nature" has a recent online article about peer review and the trial-by-fire that scientific papers sometimes now go through via blogger and social media commentary that can be rapid, highly critical, and not necessarily polite. The article mentions a couple of the more famous recent examples of this, and there will undoubtedly be far more cases down the road.
The traditional print-form peer review and discussion (that some favor) can be painfully slow and wearisome. As someone who thinks the new sort of 'open science' internet debate is therefore mostly to the good, and certainly the wave of the future, I find this 'fast feedback' over the Web a positive and irreversible development, even though it can strain nerves/feelings.

What I found especially interesting in the "Nature" piece was the following admission that math and physics have already long dealt with (if not even relished) rapid (and early) feedback, and it is primarily the biological sciences where most resistance lies:
"In some fields, notably mathematics and physics, this sort of public discourse on a paper has long been the norm, both before and after publication. Most researchers in those fields have been depositing their draft papers in the preprint server for two decades. And when blogging became popular around the turn of the millennium, they were quick to start debating their research in that form.

"Scientists in other fields seem less willing to get involved in pre-publication discussion. Biologists, in particular, are notoriously reluctant to publicly discuss their own work or comment on the work of others for fear of being scooped by competitors or of offending future reviewers of their own work. Adding to the disincentive is the knowledge that tenure committees and funding agencies do not explicitly reward online activity."
In general, biologists ('notorious' indeed! ;-)) often appear both more critical of one another, and more sensitive to criticism, than physicists and math-types tend to be, and thus are naturally more fearful of the kind of wrangling free-for-all the open Web offers. Anyway, food for thought....


rpg said...

Hi Shecky

Thanks for commenting at OT!

Me and a few people closer to the coalface had a discussion on twitter yesterday, and it turns out that the phrase "Most researchers in those fields have been depositing their draft papers in the preprint server for two decades" is so much bullshit. That's only true in certain disciplines, so I think the reporter failed in her portrayal, there.

Shecky R. said...

Yes, the word "Most" is likely an exaggeration, but I think the point the writer was trying to make is that arXive-like (and even other collaborative-type) sites have been used longer, more effectively, and by more participants in the physical sciences than in the life sciences where there's a greater tendency toward 'turf' battles (I happen to think that's a valid point, though others will argue).
[p.s... it was actually viewing some of the twitter discussion that caused me to put that post up, even though it wasn't usual fodder for this blog ;-) ]