Neural architecture search based on dual attention mechanism for image classification

Abstract

Deep learning neural networks based on the manual design for image classification tasks usually require a large amount of a priori knowledge and experience from experts; thus, research on designing neural network architectures automatically has been widely performed. The neural architecture search (NAS) method based on the differentiable architecture search (DARTS) ignores the interrelationships within the searched network architecture cells. The optional operations in the architecture search space lack diversity, and the large parametric and non-parametric operations in the search space make the search process inefficient. We propose a NAS method based on a dual attention mechanism (DAM-DARTS). An improved attention mechanism module is introduced to the cell of the network architecture to deepen the interrelationships between the important layers within the architecture by enhancing the attention between them, which improves the accuracy of the architecture and reduces the architecture search time. We also propose a more efficient architecture search space by adding attention operations to increase the complex diversity of the searched network architectures and reduce the computational cost consumed in the search process by reducing non-parametric operations. Based on this, we further analyze the impact of changing some operations in the architecture search space on the accuracy of the architectures. Through extensive experiments on several open datasets, we demonstrate the effectiveness of the proposed search strategy, which is highly competitive with other existing neural network architecture search methods.

Keywords

• neural architecture search
• deep learning
• image classification
• darts
• attention