A risk of bias instrument for non-randomized studies of exposures: A users' guide to its application in the context of GRADE

Rebecca L. Morgan; Kristina A. Thayer; Nancy Santesso; Alison C. Holloway; Robyn Blain; Sorina E. Eftim; Alexandra E. Goldstone; Pam Ross; Mohammed Ansari; Elie A Akl; Tommaso Filippini; Anna Hansell; Joerg J. Meerpohl; Reem A. Mustafa; Jos Verbeek; Marco Vinceti; Paul Whaley; Holger J. Schünemann

Environment International (Jan 2019)

A risk of bias instrument for non-randomized studies of exposures: A users' guide to its application in the context of GRADE

Rebecca L. Morgan,
Kristina A. Thayer,
Nancy Santesso,
Alison C. Holloway,
Robyn Blain,
Sorina E. Eftim,
Alexandra E. Goldstone,
Pam Ross,
Mohammed Ansari,
Elie A Akl,
Tommaso Filippini,
Anna Hansell,
Joerg J. Meerpohl,
Reem A. Mustafa,
Jos Verbeek,
Marco Vinceti,
Paul Whaley,
Holger J. Schünemann

Affiliations

Rebecca L. Morgan: Department of Health Research Methods, Evidence, and Impact, McMaster University, Health Sciences Centre, Room 2C14, 1280 Main Street West, Hamilton, ON L8S 4K1, Canada
Kristina A. Thayer: Integrated Risk Information System (IRIS) Division, National Center for Environmental Assessment (NCEA), Office of Research and Development, US Environmental Protection Agency, Building B (Room 211i), Research Triangle Park, NC 27711, USA
Nancy Santesso: Department of Health Research Methods, Evidence, and Impact, McMaster University, Health Sciences Centre, Room 2C14, 1280 Main Street West, Hamilton, ON L8S 4K1, Canada
Alison C. Holloway: Department of Obstetrics and Gynecology, McMaster University, Health Sciences Centre, Room 3N52A, 1280 Main Street West, Hamilton, ON L8S 4K1, Canada
Robyn Blain: ICF International Inc., 9300 Lee Highway, Fairfax, VA, USA
Sorina E. Eftim: ICF International Inc., 9300 Lee Highway, Fairfax, VA, USA
Alexandra E. Goldstone: ICF International Inc., 9300 Lee Highway, Fairfax, VA, USA
Pam Ross: ICF International Inc., 9300 Lee Highway, Fairfax, VA, USA
Mohammed Ansari: School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, ON K1H 8M5, Canada
Elie A Akl: Department of Health Research Methods, Evidence, and Impact, McMaster University, Health Sciences Centre, Room 2C14, 1280 Main Street West, Hamilton, ON L8S 4K1, Canada; Department of Internal Medicine, Faculty of Health Sciences, American University of Beirut, P.O. Box: 11-0236, Riad-El-Solh Beirut 1107 2020, Lebanon
Tommaso Filippini: Department of Biomedical, Metabolic and Neural Sciences, University of Modena and Reggio Emilia, Italy
Anna Hansell: MRC-PHE Centre for Environment and Health, Imperial College London, St Mary's Campus, Praed St, Paddington, London W2 1PG, UK; Public Health Directorate, Imperial College Healthcare NHS Trust, St Mary's Hospital, Paddington, London, W2 1PG, UK; Centre for Environmental Health and Sustainability, University of Leicester, George Davies Building, University Road, Leicester LE1 7RH, UK
Joerg J. Meerpohl: Institute for Evidence in Medicine (for Cochrane Germany Foundation), Medical Center - University of Freiburg, Breisacher Strasse 153, 79110 Freiburg, Germany
Reem A. Mustafa: Department of Health Research Methods, Evidence, and Impact, McMaster University, Health Sciences Centre, Room 2C14, 1280 Main Street West, Hamilton, ON L8S 4K1, Canada; Division of Nephrology and Hypertension, Department of Medicine, University of Kansas Medical Center, 3901 Rainbow Boulevard, Kansas City, KS 66160, USA
Jos Verbeek: Finnish Institute of Occupational Health, Cochrane Work, Neulaniementie 4, 70701 Kuopio, Finland
Marco Vinceti: Department of Biomedical, Metabolic and Neural Sciences, University of Modena and Reggio Emilia, Italy; Department of Epidemiology, Boston University School of Public Health, Boston, MA, USA
Paul Whaley: Lancaster Environment Centre, Lancaster University, Lancaster LA1 4YQ, UK
Holger J. Schünemann: Department of Health Research Methods, Evidence, and Impact, McMaster University, Health Sciences Centre, Room 2C14, 1280 Main Street West, Hamilton, ON L8S 4K1, Canada; Department of Medicine, McMaster University, Health Sciences Centre, Room 2C14, 1280 Main Street West, Hamilton, ON L8S 4K1, Canada; Corresponding author at: Department of Health Research Methods, Evidence and Impact, Health Sciences Centre, Room 2C14, 1280 Main Street West, Hamilton, ON L8S 4K1, Canada.

Journal volume & issue: Vol. 122
pp. 168 – 184

Abstract

Read online

The objective of this paper is to explain how to apply, interpret, and present the results of a new instrument to assess the risk of bias (RoB) in non-randomized studies (NRS) dealing with effects of environmental exposures on health outcomes. This instrument is modeled on the Risk Of Bias In Non-randomized Studies of Interventions (ROBINS-I) instrument. The RoB instrument for NRS of exposures assesses RoB along a standardized comparison to a randomized target experiment, instead of the study-design directed RoB approach. We provide specific guidance for the integral steps of developing a research question and target experiment, distinguishing issues of indirectness from RoB, making individual-study judgments, and performing and interpreting sensitivity analyses for RoB judgments across a body of evidence. Also, we present an approach for integrating the RoB assessments within the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) framework to assess the certainty of the evidence in the systematic review. Finally, we guide the reader through an overall assessment to support the rating of all domains that determine the certainty of a body of evidence using the GRADE approach. Keywords: Risk of bias, Environmental health, GRADE, Non-randomized studies, Study limitations, ROBINS

Published in Environment International

ISSN: 0160-4120 (Print)
Publisher: Elsevier
Country of publisher: United Kingdom
LCC subjects: Geography. Anthropology. Recreation: Environmental sciences
Website: https://www.journals.elsevier.com/environment-international

About the journal