IEEE Access (Jan 2024)
A Mathematically Inspired Meta-Heuristic Approach to Parameter (Weight) Optimization of Deep Convolution Neural Network
Abstract
Deep Neural Learning has unequivocally demonstrated its exceptional capabilities, surpassing the performance of various Machine Learning techniques in recent years. Optimizing the parameters, such as weights, in Deep Neural Learning can be a challenging and time-consuming process due to its computational complexity. While Gradient Descent (GD) is presently the most widely used technique for weight optimization in alliance with the backpropagation (BP) technique, its limitations, such as slow convergence and susceptibility to local minima, have been extensively documented. Other than the inherent issues mentioned with derivative-based methods, many problems are characterized as non-convex, discontinuous, and non-differentiable leading to problems of vanishing gradient. Addressing these challenges, we explore alternative optimization approaches by harnessing the capabilities of Meta-Heuristic Algorithms (MH). In its generic form, MH algorithms struggle to preserve variation among offspring, which can result in premature convergence of the solutions. Additionally, the computational overhead MH Algorithms often leads to inefficient optimization of high-dimensional problems. Hence, our investigation explores the application of MH-based novel methods for optimizing the Deep Neural Network’s weight. Further, this exploration aims to overcome the inherent drawbacks of BP and GD techniques and generic MH algorithms. With this research, we strive to present a compelling insight into the potential advancements achievable through novel variations of MH algorithms in Deep Convolutional Neural Network parameter (weight) optimization. We ere able to demonstrate accuracy improvement of traditional GACNN 85.27% to Novel Meta Heuristic based CNN optimizer to 98.7% with reduced computational complexity.
Keywords
