A Convolutional Transformer Model for Multivariate Time Series Prediction

Abstract

This paper presents a multivariate time series prediction framework based on a transformer model consisting of convolutional neural networks (CNNs). The proposed model has a structure that extracts temporal features of input data through CNN and interprets correlations between variables through an attention mechanism. This framework solves the problem of the inability to simultaneously analyze the temporal features of the input data and the correlation between variables, which is a limitation of the forecasting models presented in existing studies. We designed a forecasting experiment using several time series datasets with various data characteristics to precisely evaluate the proposed model. In addition, comparative experiments were performed between the proposed model and several predictive models proposed in recent studies. Furthermore, we conducted ablation studies on the extent to which the proposed CNN structure in the prediction model affects the forecasting results by substituting a specific layer of the model. The results of the experiments showed that the proposed predictive model exhibited good performance in predicting time series data with a clear cycle and high correlation between variables, and improved the accuracy by approximately 3% to 5% compared with that of previous studies’ time series prediction models.

Keywords

• Artificial neural networks
• predictive models
• time series prediction