Archive by Author | kmmarasi

Peer-review

Peer review is an important part of publication and funding dissemination. Funding applications are evaluated by reviewers who are experts in the same field and their comments and recommendations are used to make the final funding decision. The goal of peer review is to ensure quality and excellence in scientific research.1 The principles of peer review include the absence of conflict of interest, fairness and effective evaluation.1 However there has been recent debates on the value of peer review for quite some time. Some of the issues that have been raised are (i) too much focus on applicants’ reputation,2 (ii) personal biases,3 and (iii) reliability of the decisions made by panelists.

Both Moran and the Nature article “the debate about peer review” point out that peer- review focuses too much on applicants’ reputation.3; 4 The concern here is that proposals with famous and established scientists or elite institutions associated with them will get funded, possibly leaving out novel promising research by starting researchers. While the evaluation of prior success of applicants is part of identifying the most deserving applicant when making decision about providing funding – just as for a job opportunity, it is a concern if prior achievements or seniority of an applicant dominate the quality and potential of the research that is being considered for funding. Moran also suggests that peer review may not be very good at identifying the really path-breaking research. She suggests that maybe it is unintentionally ruling out the risky projects that have the potential to really change the field of research.4 While she states this as a real concern, I find it quite difficult to suggest a solution to this problem. How do we ensure that peer- review is good at identifying path-breaking research? The reviewers are experts in the research field hence they are expected and trained to be able to identify research with greater potential. If their training and expertise are somehow not meeting the expectations of their job (i.e. identifying research with greater potential) perhaps a different and more improved approach to training research panelists is encouraged.

Another critique she points out to is that sometimes the review committees are not reading the details of the proposals when choosing the most promising research. While such practice questions the reliability of the decisions being made by the review panel, I believe it is easily fixable by enforcing strict regulation on the review process such as by involving a judge or a senior/highly respected individual to supervise the review process. Lamont further notes flaws of peer review such as final decisions on funding being influenced by reviewers’ personal interests, and surprising deal making among panelists to prioritize their personal plans (i.e. rushing a final decision so that they could catch a plane on time).2 While reviewers may find projects that relate to their field of research exciting, the aim of peer review is to eliminate such personal biases to make informed decisions on which project deserves funding. Perhaps it is best to have a set of questions that each reviewer must answer for each application such as identifying the short term and long terms benefits of the research to the target population/society and together as a committee evaluate and prioritize timely research supported by evidence-based research. While it may be difficult to completely eliminate personal biases, having such requirements to support each reviewer’s decision may make it difficult to fund a famous and well-established researcher over a starting researcher who may have promising research ideas.

While it has been challenging to answer the question of how to improve the process of peer-review, I suggest a few ways to minimize currently identified flaws of peer-review. To avoid funding projects based on applicants’ reputation or researchers’ personal interests, my suggestion is to blind the reviewers to applicants’ information until most promising research projects have been chosen based on their scholarly merit, quality of proposed research, and methodology etc. Once such a list has been created, applicants’ prior successes and achievements could be evaluated to identify the most deserving applicant. I also support Walsh et al.’s theory of opening up the review process so the reviewers are held accountable for their decisions.3

References

Canadian Institutes of Health Research (CIHR). (2013). Peer review: Overview.   Retrieved October 28, 2015, from http://www.cihr-irsc.gc.ca/e/39380.html

Jaschik, S. (2009). The ‘Black Box’ of Peer Review.   Retrieved October 28, 2015, from https://www.insidehighered.com/news/2009/03/04/peerreview

SciBytes. (2013). The Debate about Peer Review.   Retrieved October 28, 2015, from http://www.nature.com/scitable/blog/scibytes/the_debate_about_peer_review

Moran, B. (2015). Does Peer Review Really Work?   Retrieved October 28, 2015, from http://www.bu.edu/research/articles/does-peer-review-really-work/

Advertisements