Uncertainty analysis: an evaluation metric for synthesis science

Abstract

The methods for conducting reductionist ecological science are well known and widely used. In contrast, those used in the synthesis of ecological science (i.e., synthesis science) are still being developed, vary widely, and often lack the rigor of reductionist approaches. This is unfortunate because the synthesis of ecological parts into a greater whole is critical to understanding many of the environmental challenges faced by society. To help address this imbalance in approaches, we examine how the rigor of ecological synthesis science might be increased by using uncertainty as an evaluation metric—as a parallel to methods used in reductionist science. To estimate and understand uncertainty we propose that it be divided into four general classes: (1) measurement uncertainty (i.e., experimental error) as defined by precision and accuracy, (2) sampling uncertainty that reflects natural variation in space and time as quantified by classical statistical moments (e.g., mean and variance), (3) model prediction uncertainty which relates to the transformation of measurements into other variables of interest (e.g., plant dimensions to biomass), and (4) model selection uncertainty which relates to uncertainty about the form of the relationships used in models. Of these sources of uncertainty, model selection is the least understood and potentially, the most important, because it is integral to how components of a system are combined and it reflects imperfect knowledge about these relationships. To demonstrate uncertainty in synthesis science, we examine each source of uncertainty in an analysis that estimates the live tree biomass of a forest and how knowledge of each source can improve future estimates. By quantifying sources of uncertainty in synthesis science, it should be possible to make rigorous comparisons among results, to judge whether they differ within the bounds of measurement and knowledge, and to assess the degree to which scientific progress is being made. However, to be accepted as a standard method, best practices analogous to those used in reductionist science need to be developed and implemented.

Keywords

• aboveground biomass
• H. J. Andrews Experimental Forest
• measurement uncertainty
• model parameter uncertainty
• model selection uncertainty
• Oregon, USA