Frontiers in Computational Neuroscience (Apr 2012)
Learning from open source software projects to improve scientific review
Abstract
Peer-reviewed publications are the primary mechanism for sharing scientific results. The current peer-review process is, however, fraught with many problems that undermine the pace, validity, and credibility of science. We highlight five salient problems: (1) Reviewers are expected to have comprehensive expertise; (2) Reviewers do not have sufficient access to methods and materials to evaluate a study; (3) Reviewers are not acknowledged; (4) There is no measure of the quality of a review; and (5) Reviews take a lot of time, and once submitted cannot evolve. We propose that these problems can be resolved by making the following changes to the review process. Distributing reviews to many reviewers would allow each reviewer to focus on portions of the article that reflect the reviewer’s specialty or area of interest and place less of a burden on any one reviewer, enabling a more comprehensive and timely review. Providing reviewers materials and methods to perform comprehensive evaluation would facilitate transparency, replication of results and enable greater scrutiny by people from different fields using different nomenclature, leading to greater clarity and cross-fertilization of ideas. Acknowledging reviewers makes it possible to quantitatively assess reviewer contributions, which could be integrated with assessments for promotions and grants. Quantifying review quality could help establish the importance of reviewers and information generated during a review, and assess the importance of a submitted article. Finally, we recommend expediting post-publication reviews and allowing for the dialogue to continue and flourish in a dynamic and interactive manner. We argue that these solutions can be addressed by building upon computer programming code management systems. In this article, we provide examples of current code review systems that offer opportunities for addressing the above problems, and offer suggestions for enhancing code review systems for article review.
Keywords
