Development of the Scientific, Transparent and Applicable Rankings (STAR) tool for clinical practice guidelines

Nan Yang; Hui Liu; Wei Zhao; Yang Pan; Xiangzheng Lyu; Xiuyuan Hao; Xiaoqing Liu; Wen'an Qi; Tong Chen; Xiaoqin Wang; Boheng Zhang; Weishe Zhang; Qiu Li; Dong Xu; Xinghua Gao; Yinghui Jin; Feng Sun; Wenbo Meng; Guobao Li; Qijun Wu; Ze Chen; Xu Wang; Janne Estill; Susan L. Norris; Liang Du; Yaolong Chen; Junmin Wei; Xiuyuan Hao

doi:10.1097/CM9.0000000000002713

Chinese Medical Journal (Jun 2023)

Development of the Scientific, Transparent and Applicable Rankings (STAR) tool for clinical practice guidelines

Nan Yang,
Hui Liu,
Wei Zhao,
Yang Pan,
Xiangzheng Lyu,
Xiuyuan Hao,
Xiaoqing Liu,
Wen'an Qi,
Tong Chen,
Xiaoqin Wang,
Boheng Zhang,
Weishe Zhang,
Qiu Li,
Dong Xu,
Xinghua Gao,
Yinghui Jin,
Feng Sun,
Wenbo Meng,
Guobao Li,
Qijun Wu,
Ze Chen,
Xu Wang,
Janne Estill,
Susan L. Norris,
Liang Du,
Yaolong Chen,
Junmin Wei,
Xiuyuan Hao

Affiliations

Nan Yang: 1Evidence-Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou, Gansu 730000, China;
Hui Liu: 2Institute of Global Health, University of Geneva, Geneva 1202, Switzerland;
Wei Zhao: 4Editor-in-Chief Office, Chinese Medical Association Publishing House, Beijing 100052, China;
Yang Pan: 5Marketing Department, Chinese Medical Association Publishing House, Beijing 100052, China;
Xiangzheng Lyu: 6Editorial Department, National Medical Journal of China, Chinese Medical Association Publishing House, Beijing 100052, China;
Xiuyuan Hao: 7Editorial Department, Chinese Medical Journal, Chinese Medical Association Publishing House, Beijing 100052, China;
Xiaoqing Liu: 8Division of Infectious Diseases, Department of Internal Medicine, State Key Laboratory of Complex Severe and Rare Disease, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing 100730, China;
Wen'an Qi: 9Editorial office of Digital Medicine and Health, Chinese Medical Association Publishing House, Beijing 100052, China;
Tong Chen: 10Department of Hematology, Huashan Hospital, Fudan University, Shanghai 200040, China;
Xiaoqin Wang: 11Evidence-based Medicine Center, Fudan University, Shanghai 200032, China;
Boheng Zhang: 12Department of Hepatic Oncology, Xiamen Clinical Research Center for Cancer Therapy, Zhongshan Hospital (Xiamen), Fudan University, Xiamen, Fujian 361015, China;
Weishe Zhang: 13Department of Obstetrics, Xiangya Hospital, Central South University, Changsha, Hunan 410008, China;
Qiu Li: 14Department of Nephrology, Children's Hospital of Chongqing Medical University, National Clinical Research Center for Child Health and Disorders, Ministry of Education Key Laboratory of Child Development and Disorders, Chongqing Key Laboratory of Pediatrics, Chongqing 400014, China;
Dong Xu: 15SMU Institute for Global Health (SIGHT), School of Health Management and Dermatology Hospital, Southern Medical University (SMU), Guangzhou, Guangdong 510515, China;
Xinghua Gao: 16Department of Dermatology, The First Hospital of China Medical University, Shenyang, Liaoning 110001, China;
Yinghui Jin: 17Center for Evidence-Based and Translational Medicine, Zhongnan Hospital of Wuhan University, Wuhan, Hubei 430071, China;
Feng Sun: 18School of Public Health, Peking University, Beijing 100191, China;
Wenbo Meng: 19Department of General Surgery, The First Hospital of Lanzhou University, Lanzhou, Gansu 730000, China;
Guobao Li: 20Department of Lung Disease, Shenzhen Third People's Hospital, The Second Affiliated Hospital, Southern University of Science and Technology, Shenzhen, Guangdong 518112, China;
Qijun Wu: 21Department of Clinical Epidemiology, Shengjing Hospital of China Medical University, Shenyang, Liaoning 110004, China;
Ze Chen: 1Evidence-Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou, Gansu 730000, China;
Xu Wang: 22Chevidence Lab of Child and Adolescent Health, Children's Hospital of Chongqing Medical University, Chongqing 400014, China;
Janne Estill: 1Evidence-Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou, Gansu 730000, China;
Susan L. Norris: 23Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, Oregon 97239, USA;
Liang Du: 24Chinese Evidence-Based Medicine Center, West China Hospital, Sichuan University, Chengdu, Sichuan 610041, China;
Yaolong Chen: 1Evidence-Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou, Gansu 730000, China;
Junmin Wei: 27Chinese Medical Association Publishing House, Beijing 100052, China.
Xiuyuan Hao

DOI: https://doi.org/10.1097/CM9.0000000000002713
Journal volume & issue: Vol. 136, no. 12
pp. 1430 – 1438

Abstract

Read online

Abstract. Background:. This study aimed to develop a comprehensive instrument for evaluating and ranking clinical practice guidelines, named Scientific, Transparent and Applicable Rankings tool (STAR), and test its reliability, validity, and usability. Methods:. This study set up a multidisciplinary working group including guideline methodologists, statisticians, journal editors, clinicians, and other experts. Scoping review, Delphi methods, and hierarchical analysis were used to develop the STAR tool. We evaluated the instrument's intrinsic and interrater reliability, content and criterion validity, and usability. Results:. STAR contained 39 items grouped into 11 domains. The mean intrinsic reliability of the domains, indicated by Cronbach's α coefficient, was 0.588 (95% confidence interval [CI]: 0.414, 0.762). Interrater reliability as assessed with Cohen's kappa coefficient was 0.774 (95% CI: 0.740, 0.807) for methodological evaluators and 0.618 (95% CI: 0.587, 0.648) for clinical evaluators. The overall content validity index was 0.905. Pearson's r correlation for criterion validity was 0.885 (95% CI: 0.804, 0.932). The mean usability score of the items was 4.6 and the median time spent to evaluate each guideline was 20 min. Conclusion:. The instrument performed well in terms of reliability, validity, and efficiency, and can be used for comprehensively evaluating and ranking guidelines.

Published in Chinese Medical Journal

ISSN: 0366-6999 (Print); 2542-5641 (Online)
Publisher: Wolters Kluwer
Country of publisher: United Kingdom
LCC subjects: Medicine
Website: https://journals.lww.com/cmj/pages/default.aspx

About the journal