Global Epidemiology (Nov 2020)
Systematically evaluating and integrating evidence in National Ambient Air Quality Standards reviews
Abstract
As part of the review process for the National Ambient Air Quality Standards (NAAQS), the United States Environmental Protection Agency (US EPA) conducts systematic reviews and assesses causal relationships between air pollutant exposures and human health effects using a framework it developed specifically for this purpose. We demonstrate how this framework could be improved by adding transparent criteria for assessing study quality, as well as detailed methods for integrating evidence in a way that fully and systematically considers individual study quality and relevance, and the coherence of results across studies within and across scientific disciplines. For example, the framework can include not just a list of study quality aspects for evaluating human and animal studies, but also aspects for evaluating in vitro studies. In addition, for all realms of evidence, the framework can specify the criteria for each study quality aspect that must be met to demonstrate that a study is of high quality. These aspects can be considered in a transparent and systematic fashion for each study, with the quality evaluations forming the basis for weighing evidence as it is integrated within and across disciplines, and ultimately for reaching conclusions regarding causality. The human relevance of experimental evidence can also be considered, particularly with respect to studies that evaluate upstream events vs. apical effects, and to how informative these studies are for interpreting epidemiology study results. As it is the goal of all regulations, including the NAAQS, to be based on sound science, these additions to the NAAQS systematic review and casual determination framework will make NAAQS causality assessments more transparent and reflective of the scientific evidence, and will allow for scientifically defensible decision-making. The modified framework can also be applied to systematic reviews of other substances by risk assessors and regulatory agencies around the world.
