Evolution of appraisal tool usage preferences in PROSPERO records: a study of non-Cochrane systematic reviews

Abstract

Abstract Objectives This research-on-research substudy uses a data-driven approach to investigate the range of appraisal tools in non-Cochrane systematic reviews and meta-analyses registered in the International Prospective Register of Systematic Reviews (PROSPERO). Study design and setting A comprehensive web scraping of all completed non-Cochrane registrations in PROSPERO from February 2011 to December 2017 was performed. The focus was classifying the appraisal tools based on study type, assessment aspects, and research topics. Results After analyzing 17,708 complete records, we found a predominant use of methodological quality assessment tools compared to those for reporting quality or risk of bias (RoB). This indicates a greater emphasis on methodological rigor in the studied protocols. Various tools for assessing methodological quality were observed, reflecting the complexity of such evaluations. Instruments designed for evaluating methodological or reporting quality were mainly intended for non-randomized clinical trials or observational studies, unlike RoB tools more commonly used in randomized clinical trials. No distinct trends in tool usage were observed in specific research conditions or domains, suggesting that tool choice is influenced more by study design than research topic. Conclusion This study provides insights into the preferential use of various assessment tools in conducting non-Cochrane systematic reviews, as evidenced in PROSPERO records. The findings reveal various methodological assessment tools, underscoring their versatility across different study designs and research areas.

Keywords

• Research-on-research
• Methodological appraisal tools
• Protocols
• Systematic reviews
• Methodological quality
• PROSPERO repository