Deep context-attentive transformer transfer learning for financial forecasting

Abstract

This study presents 2CAT (CNN-Correlation-based Attention Transformer), a deep learning model for financial time-series forecasting. The model integrates signal decomposition, convolutional layers, and correlation-based attention mechanisms to capture temporal patterns. A transfer learning framework is incorporated to enhance generalization across markets through pretraining, encoder freezing, and fine-tuning. Evaluation on six stock indices—Dow Jones Industrial Average (DJIA), Nikkei 225 (N225), Hang Seng Index (HSI), Shanghai Stock Exchange (SSE), Bombay Stock Exchange (BSE), and the Stock Exchange of Thailand (SET)—demonstrates strong predictive accuracy. On DJIA, 2CAT records an MSE of 0.0655, MAE of 0.2023, and R2 of 0.9169, outperforming Deep-Transformer, which yields an MSE of 0.1360 and R2 of 0.8274. The SET index, which posed challenges for previous models, demonstrates notable improvement with 2CAT, achieving an R2 of 0.9094. Wilcoxon signed-rank test confirms statistically significant gains in non-transfer learning scenarios at the 0.05 level. Transfer learning experiments reveal statistically significant improvements, reinforcing the feasibility of cross-market knowledge transfer. An ablation study highlights the impact of architectural refinements and rotary positional encoding, while prediction horizon analysis confirms stable forecasting performance. These results establish 2CAT as a robust financial forecasting framework adaptable to diverse market conditions.

Keywords

• Knowledge transfer
• Transformer-based model
• Feature extraction
• Contextual learning
• Temporal forecasting