Effects of Learning Rates and Optimization Algorithms on Forecasting Accuracy of Hourly Typhoon Rainfall: Experiments With Convolutional Neural Network

Abstract

Abstract The current study used seven optimization algorithms such as stochastic gradient descent (SGD), root mean square propagation (RMSprop), adaptive grad (AdaGrad), adaptive delta (AdaDelta), adaptive moment estimation (Adam), adaptive maximum (Adamax), and Nesterov‐accelerated adaptive moment estimation (Nadam) and eight learning rates (1, 0.1, 0.01, 0.001, 0.0001, le‐05, le‐06, and le‐07) to investigate the effects of these learning rates and optimizers on the forecasting performance of the convolutional neural network (CNN) model to forecast hourly typhoon rainfall. The model was developed using antecedent hourly typhoon rainfall within a 500 km radius from each typhoon center. Results showed that too‐large and too‐small learning rates would result in the inability of the model to learn anything to forecast hourly typhoon rainfall. The CNN model showed the best performance for learning rates of 0.1, 0.01, and 0.001 to forecast hourly typhoon rainfall. For long‐lead‐time forecasting (1–6 hr), the CNN model with SGD, RMSprop, AdaGrad, AdaDelta, Adam, Adamax, Nadam optimizers and learning rates of 0.1, 0.01, and 0.001 showed more accurate forecasts than the existing models. Therefore, this study recommends that future work may consider the CNN model as an alternative to the existing model for disaster warning systems.