Accounting for observation noise in equation‐free forecasting: The hidden‐Markov S‐map

Abstract

Abstract Ecosystems contain numerous species interacting with each other and the environment. However, data on all relevant state variables is rarely available, hampering inference and prediction. Empirical dynamic modelling (EDM) is a valuable tool for prediction, inference and control in such partially observed systems. However, EDM typically assumes that the available time series are observed without error. Failing to account for observation noise strongly biases estimates of Lyapunov exponents and reduces forecast accuracy. To address this limitation, we propose incorporating EDM into a hidden Markov framework and using an iterative scheme based on the expectation maximization (EM) algorithm to obtain filtered state and parameter estimates. We evaluate the performance of this approach on several simulated dynamical systems with a range of additive noise levels, as well as on insect population time series. Accounting for observation noise improved accuracy of population forecasts and estimates of Lyapunov exponents (LE) over a wide range of noise levels relevant to ecological time series.

Keywords

• empirical dynamic modelling
• hidden‐Markov
• S‐map