Hyperparameter Optimization of CNN for Map Building

Abstract

This article describes an approach for solving the task of finding hyperparameters of an artificial neural network, which is used for making a 2D land map. The main goal of research was an analysis of methods for finding hyperparameters and creating a better method for solving this task, which would be based on existing methods. We considered on various hyperparameters such as velocity of training, coefficient of regularization, size of batch, probability of drop out, shifting, used for batch normalization. Among existing methods for finding hyperparameters we considered on the random search method, searching by grid, the Bayesian optimization, the evolution algorithm, the optimization, based on gradients, and the spectral method. As a result, we created a new method for finding hyperparameters which showed a better result in most of the use cases, which we have (mostly for middle European part of Russia). The main idea of the method for finding hyperparameters is consisted in an approach for optimization of the quality function with a simple condition for lower and upper limits and a demand that the value of the function needed to be an integer number. This task may be solved with a simple genetic algorithm. Using the optimization algorithm without evaluating derivatives gives decreasing time complexity of the algorithm without losing quality of the algorithm. In many cases the quality of result was better than results of existing methods.

Keywords

• convolutional neural network
• grid search
• random search
• evolutionary optimization
• bayesian optimization
• gradient based optimization
• spectral method