Adaptive Levenberg-Marquardt Algorithm Based Echo State Network for Chaotic Time Series Prediction

Abstract

Echo state networks (ESNs) have wide applications in chaotic time series prediction. In the ESN, if the smallest singular value of the reservoir state matrix is infinitesimal, the ill-posed problem might occur during the training process. To overcome this problem, an adaptive Levenberg-Marquardt (LM) algorithm-based echo state network (ALM-ESN) is developed. In the developed ALM-ESN, a new adaptive damping term is introduced into the LM algorithm. The adaptive factor is amended by the trust region technique, furthermore, convergence analysis, and stability analysis are performed. Moreover, to make the inputs fall within the active region of the activation function and improve the learning speed, a weight initialization method using linear algebra is deployed to determine the appropriate input weights and reservoir weights. Simulations demonstrate that the ALM-ESN can overcome the ill-posed problem. Furthermore, it exhibits better performance and robustness for chaotic time series prediction than some other existing methods.

Keywords

• Echo state network
• adaptive Levenberg-Marquardt algorithm
• trust region technique
• weight initialization
• chaotic time series prediction