An ODE based neural network approach for PM2.5 forecasting

Abstract

Abstract Predicting time-series data is inherently complex, spurring the development of advanced neural network approaches. Monitoring and predicting PM2.5 levels is especially challenging due to the interplay of diverse natural and anthropogenic factors influencing its dispersion, making accurate predictions both costly and intricate. A key challenge in predicting PM2.5 concentrations lies in its variability, as the data distribution fluctuates significantly over time. Meanwhile, neural networks provide a cost-effective and highly accurate solution in managing such complexities. Deep learning models like Long Short-Term Memory (LSTM) and Bidirectional LSTM (BiLSTM) have been widely applied to PM2.5 prediction tasks. However, prediction errors increase as the forecasting window expands from 1 to 72 hours, underscoring the rising uncertainty in longer-term predictions. Recurrent Neural Networks (RNNs) with continuous-time hidden states are well-suited for modeling irregularly sampled time series but struggle with long-term dependencies due to gradient vanishing or exploding, as revealed by the ordinary differential equation (ODE) based hidden state dynamics–regardless of the ODE solver used. Continuous-time neural processes, defined by differential equations, are limited by numerical solvers, restricting scalability and hindering the modeling of complex phenomena like neural dynamics–ideally addressed via closed-form solutions. In contrast to ODE-based continuous models, closed-form networks demonstrate superior scalability over traditional deep-learning approaches. As continuous-time neural networks, Neural ODEs excel in modeling the intricate dynamics of time-series data, presenting a robust alternative to traditional LSTM models. We propose two ODE-based models: a transformer-based ODE model and a closed-form ODE model. Empirical evaluations show these models significantly enhance prediction accuracy, with improvements ranging from 2.91 to 14.15% for 1-hour to 8-hour predictions when compared to LSTM-based models. Moreover, after conducting the paired t-test, the RMSE values of the proposed model (CCCFC) were found to be significantly different from those of BILSTM, LSTM, GRU, ODE-LSTM, and PCNN,CNN-LSSTM. This implies that CCCFC demonstrates a distinct performance advantage, reinforcing its effectiveness in hourly PM2.5 forecasting.