Improved Search in Neuroevolution Using a Neural Architecture Classifier With the CNN Architecture Encoding as Feature Vector

Abstract

Designing Convolutional Neural Networks (CNNs) for a specific task requires not only Deep Learning expertise but also knowledge of the problem. The goal of Neuroevolution is to find CNN architectures automatically through evolution. The search time, however, is a critical problem in Neuroevolution since multiple CNNs must be trained in the evolutionary process. In this work, we propose a Neural Architecture Classifier (NAC) to avoid training architectures that would not have good performance, based on knowledge of previously trained architectures. The NAC evaluates each CNN using the CNN architecture encoding as the input. A genetic algorithm (GA) is used for evolution, and a search space with few restrictions; hence, CNN architectures can have any width, depth or shape as well as the suitable number of skip connections. We applied this methodology to solving pattern recognition problems on six well-known datasets: five subsets of the MNIST-Variations (MNIST-V) dataset and the CIFAR-10 dataset. Experiments demonstrated that integrating the NAC in the GA reduces the search time by up to 44% compared to conventional GAs. Additionally, the architectures found by evolution with our classifier achieved a better performance (4.6% on average) than those found by traditional evolution. Our results improved the state-of-the-art by 5.2% (0.33% error reduction) on the most difficult dataset of the MNIST-V (MRDBI), and an average of 4% (0.24% error reduction) on the MNIST-V datasets. Additionally, our results on CIFAR-10 are close to those of the state-of-the-art but with significantly reduced search time. The application of a classifier can be relevant and useful not only for genetic algorithms, but also for other evolutionary algorithms.

Keywords

• Convolutional neural networks
• genetic algorithms
• neuroevolution
• neural architecture classifier
• search time