The current state of uncertainty reporting in ecosystem studies: a systematic evaluation of peer‐reviewed literature

Abstract

Abstract Transparency in reporting is essential to scientific progress. No report should be considered complete without a full account of uncertainties, including those due to natural variation and measurement and model error and those incurred by handling problematic data, such as outliers. We randomly selected 132 papers published in 2019 from a list of 100 scientific journals to characterize the current state of uncertainty reporting in ecosystem studies. Each paper was evaluated for the extent to which it reported measures of uncertainty in any of four topic areas common to ecosystem studies: vegetation, soils, precipitation, and surface water. We found that most papers reported a minority of the uncertainty sources we deemed relevant. Papers on surface water reported the highest fraction of uncertainty sources (averaging 47% ± 5%), followed by soils (45% ± 4%), vegetation (32% ± 4%), and precipitation (21% ± 8%). A greater fraction of relevant uncertainty sources were reported when the topics were the primary focus of the paper (44% ± 3%) than when they were not (32% ± 4%). Sampling error—the uncertainty in replicate measurements—was the source most commonly reported in studies of vegetation (84%), soil mass (56%), and surface water (76% of papers). The source of measurement error most often reported was chemical analysis, with 41% of papers on surface water and 75% of papers on precipitation reporting this source, if applicable. In contrast, only 1 of 12 papers reporting chemistry of vegetation provided information on analytical uncertainty. Fewer papers reported what methods were used for handling missing or unusable data and observations below detection limits, but it was difficult to judge whether these sources were relevant if they were not mentioned. Finally, we found that a minority of the papers made all (21%) or some (an additional 21%) of their data available in online repositories, after correcting for a failure rate of 13% of the links. Clearly, there is room for improving the completeness and transparency of scientific reporting in ecosystem studies.

Keywords

• biomass
• ecosystem ecology
• measurement error
• model error
• precipitation
• sampling error