Exploring decision-makers’ challenges and strategies when selecting multiple systematic reviews: insights for AI decision support tools in healthcare
Beverley Shea,
Andrea C Tricco,
Clare Ardern,
Dawid Pieper,
Ba Pham,
Salmaan Kanji,
Yuan Chi,
Sera Whitelaw,
Carole Lunny,
Areti-Angeliki Veroniki,
Nicola Ferri,
Jia He (Janet) Zhang,
Jasmeen Dourka,
Emma K Reid,
Ebrahim Bagheri
Affiliations
Beverley Shea
13 University of Ottawa, Ottawa, Ontario, Canada
Andrea C Tricco
Epidemiology Division and Institute of Health Policy, Management, and Evaluation, University of Toronto Dalla Lana School of Public Health, Toronto, Ontario, Canada
Clare Ardern
15 Department of Family Practice, The University of British Columbia—Vancouver Campus, Vancouver, British Columbia, Canada
Dawid Pieper
8 Institute for Health Services and Health System Research, Faculty of Health Sciences Brandenburg, Brandenburg Medical School Theodor Fontane, Neuruppin, Brandenburg, Germany
Ba Pham
16 Knowledge Translation Program, Li Ka Shing Knowledge Institute, University of Toronto, Toronto, Ontario, Canada
Salmaan Kanji
9 Department of Pharmacy, Ottawa Hospital, Ottawa, Ontario, Canada
Yuan Chi
5 Yealth Network, Beijing Health Technology Co., Ltd, Beijing, China
Sera Whitelaw
3 Faculty of Medicine and Health Sciences, McGill University, Montreal, Québec, Canada
Carole Lunny
1 Knowledge Translation Program, Li Ka Shing Knowledge Institute, UBC, Toronto, Ontario, Canada
Areti-Angeliki Veroniki
11 Li Ka Shing Knowledge Institute of St Michael`s Hospital, Knowledge Translation Program, St Michael`s Hospital, Toronto, Ontario, Canada
Nicola Ferri
6 Department of Biomedical and Neuromotor Sciences, University of Bologna, Bologna, Italy
Jia He (Janet) Zhang
7 Anesthesiology, Pharmacology & Therapeutics, The University of British Columbia, Vancouver, British Columbia, Canada
Jasmeen Dourka
14 Knowledge Translation Program, Li Ka Shing Knowledge Institute, St Michael`s Hospital, Toronto, Ontario, Canada
Emma K Reid
4 Department of Pharmacy, Nova Scotia Health Authority, Halifax, Nova Scotia, Canada
Ebrahim Bagheri
17 Department of Electrical and Computer Engineering, Toronto Metropolitan University, Toronto, Ontario, Canada
Background Systematic reviews (SRs) are being published at an accelerated rate. Decision-makers may struggle with comparing and choosing between multiple SRs on the same topic. We aimed to understand how healthcare decision-makers (eg, practitioners, policymakers, researchers) use SRs to inform decision-making and to explore the potential role of a proposed artificial intelligence (AI) tool to assist in critical appraisal and choosing among SRs.Methods We developed a survey with 21 open and closed questions. We followed a knowledge translation plan to disseminate the survey through social media and professional networks.Results Our survey response rate was lower than expected (7.9% of distributed emails). Of the 684 respondents, 58.2% identified as researchers, 37.1% as practitioners, 19.2% as students and 13.5% as policymakers. Respondents frequently sought out SRs (97.1%) as a source of evidence to inform decision-making. They frequently (97.9%) found more than one SR on a given topic of interest to them. Just over half (50.8%) struggled to choose the most trustworthy SR among multiple. These difficulties related to lack of time (55.2%), or difficulties comparing due to varying methodological quality of SRs (54.2%), differences in results and conclusions (49.7%) or variation in the included studies (44.6%). Respondents compared SRs based on the relevance to their question of interest, methodological quality, and recency of the SR search. Most respondents (87.0%) were interested in an AI tool to help appraise and compare SRs.Conclusions Given the identified barriers of using SR evidence, an AI tool to facilitate comparison of the relevance of SRs, the search and methodological quality, could help users efficiently choose among SRs and make healthcare decisions.
