Sunday, October 7, 2012

Science With a Grain of Salt

It is becoming more and more obvious that there are serious problems with today's "science."

I'm well aware of papers that would not pass the laugh test (click here for a discussion of the most recent example, as you would guess in climate 'science') but get published anyway. 

As a person who has been a reviewer on literally dozens of papers, I have some suggestions that might help the peer-review system recover from its dismal state.
  1. Currently, the reviewers are anonymous. Fine. But, I believe the author(s) of the paper and their institution(s) and all other identifying information should be stripped from the paper. That forces the reviewers to focus on the merits of the paper and should help keep politics and political correctness to a minimum.
  2. The reviewers should have to sign a conflict of interest statement (some already have to do this) and a confidentiality statement. The latter should ban communications with any of the other reviewers as well as anyone else. If someone needs help in the review (other than from the editor of the journal who typically rides herd over the process), the review should be done by someone else. 
  3. Whenever possible, reviewers should be drawn from a variety of backgrounds. There is an upcoming conference with 40 papers. Thirty-eight of the selected presenters are from government and 36 of those are from the National Weather Service! All three of the reviewers had government backgrounds (although one is in the private sector currently). The lack of genuine diversity among reviewers tends to result in the overrepresentation of a single point of view. 
This is not meant to be a comprehensive list but simply a starting point for discussion. There is no doubt  that the peer-review system is broken. It needs to be fixed.  

No comments:

Post a Comment