AutoCyclic: Deep Learning Optimizer for Time Series Data Prediction

Abstract

Time series prediction poses a formidable challenge, marked by the inherent difficulty in capturing long-term dependencies and adapting to intricate data patterns. Existing methods, spanning statistical models and neural networks, often grapple with issues such as underfitting and overfitting. This study addresses these challenges by introducing Autocyclic Learning Rate (AutoCyclic), an innovative approach that seamlessly integrates cosine cyclic learning rates with considerations for autocorrelation and variance. AutoCyclic dynamically adjusts learning rates based on the characteristics of time series data, effectively mitigating challenges related to local minima and demonstrating robust adaptability to outliers. In evaluation across diverse datasets, including ETTm2, M4, and WindTurbine, AutoCyclic consistently outperforms traditional optimizers such as Adams Optimizer and Cosine Cyclic Learning Rate. The results underscore AutoCyclic’s superior performance, showcasing its potential as a pivotal tool for enhancing predictive modeling in various time series forecasting scenarios. The groundbreaking nature of AutoCyclic lies in its ability to address the complexities of time series prediction, providing a valuable solution to the limitations faced by existing models. The study serves as a key contribution to the ongoing research in timeseries data prediction, with implications for improving the accuracy and efficiency of predictive models in diverse applications. For those interested in implementing AutoCyclic, the code is available at https://github.com/wtfish/AutoCyclic.

Keywords

• Autocorrelation
• cosine cyclic learning rate
• deep transformer
• optimizer
• time series
• variance