The Delphi process: a solution for reviewing novel grant applications

Abstract

Cath Holliday1, Monica Robotin1,21Cancer Council New South Wales, Sydney; 2University of Sydney, School of Public Health, Sydney, New South Wales, AustraliaIntroduction: Traditional scientific review processes are not well suited for evaluating the merits of research in situations where the available scientific evidence is limited and if review panels have widely divergent opinions. This study tested whether a Delphi process is useful in grant selection.Materials and method: A Delphi process prioritized novel research proposals in pancreatic cancer. Five reviewers holding similar grants overseas ranked research applications by scientific merit, innovativeness, and level of risk.Result: Three rounds of voting evaluated the best 10 applications received. In the first round of the Delphi process, scores ranged from 5.0 to 8.3. After the second round, the cumulative scores of the eight remaining applications ranged from 10 to 12.6. At the end of the third round, the final cumulative scores of the remaining six applications ranged from 13.6 to 18.2. The four highest ranking applications were recommended for funding, with agreement from reviewers.Conclusion: A modified Delphi process proved to be an efficient, transparent, and equitable method of reviewing novel grant applications in a specialized field of research, where no local expertise was available. This process may also be useful for other peer review processes, particularly where there is limited access to local experts.Keywords: delphi process, research grant selection, consensus