Survey on Compact Neural Network Model Design

Abstract

In recent years, convolutional neural network has achieved excellent performance in a wide range of applications, but it consumes huge resources, which is a challenge for its application to mobile terminals and embedded devices. To this end, network models need to be balanced in size, speed and accuracy. Firstly, two methods of network compression and acceleration, neural network compression and compact neural network, are briefly intro-duced from the perspective of whether the model is pre-trained or not. Specifically, this paper describes the design method of compact neural network, shows the different operation modes, emphasizes the characteristics of these operations, and divides them into two categories: model design based on spatial convolution and model design based on shift convolution according to the different basic operations. Then, each class selects three network models to discuss from the basic operation unit, the core building block and the overall network structure. At the same time, the performance of each network and the conventional network on the ImageNet dataset is analyzed. Finally, this paper summarizes the existing design techniques of compact neural network and looks forward to the future develop-ment direction.

Keywords

• convolutional neural network (cnn)
• lightweight
• shift operation
• convolution method