On the assessment of the ability of measurements, nowcasts, and forecasts to track changes

Abstract

Abstract Background Measurements, nowcasts, or forecasts ideally should correctly reflect changes in the values of interest. In this article, we focus on how to assess the ability of measurements, nowcasts, or forecasts to correctly predict the direction of changes in values - which we refer to as the ability to track changes (ATC). Methods We review and develop visual techniques and quantitative measures to assess ATC. Extensions for noisy data and estimation uncertainty are implemented using bootstrap confidence intervals and exclusion areas. Results We exemplarily illustrate the proposed methods to assess the ability to track changes for nowcasting during the COVID-19 pandemic, patient admissions to an emergency department, and non-invasive blood pressure measurements. The proposed methods effectively evaluate ATC across different applications. Conclusions The developed ATC assessment methods offer a comprehensive toolkit for evaluating the ATC of measurements, nowcasts, and forecasts. These techniques provide valuable insights into model performance, complementing traditional accuracy measures and enabling more informed decision-making in various fields, including public health, healthcare management, and medical diagnostics.