The Elitist Non-Dominated Sorting Crisscross Algorithm (Elitist NSCA): Crisscross-Based Multi-Objective Neural Architecture Search

Abstract

In recent years, neural architecture search (NAS) has been proposed for automatically designing neural network architectures, which searches for network architectures that outperform novel human-designed convolutional neural network (CNN) architectures. Related research has always been a hot topic. This paper proposes a multi-objective evolutionary algorithm called the elitist non-dominated sorting crisscross algorithm (elitist NSCA) and applies it to neural architecture search, which considers two optimization objectives: the accuracy and network parameters. In the algorithm, an innovative search space borrowed from the latest residual block and dense connection is proposed to ensure the quality of the compact architectures. A variable-length crisscross optimization strategy, which creatively iterates the evolution through inter-individual horizontal crossovers and intra-individual vertical crossovers, is employed to simultaneously optimize the microstructure parameters and macroscopic architecture of the CNN. In addition, a corresponding mutation operator is added pertinently based on the performance of the proxy model, and the elitist strategy is improved through pruning to reduce the impact of abnormal fitnesses. The experimental results on multiple datasets show that the proposed algorithm has a higher accuracy and robustness than those of certain state-of-the-art algorithms.

Keywords

• neural architecture search
• crisscross optimization
• evolutionary multi-objective optimization
• image classification