Adaptive Pruning of Convolutional Neural Network

Abstract

Deep convolutional neural networks (CNNs) have attained remarkable success in numerous visual recognition tasks. There are two challenges when adopting CNNs in real-world applications: a) Existing CNNs are computationally expensive and memory intensive, impeding their use in edge computing; b) there is no standard methodology for designing the CNN architecture for the intended problem. Network pruning/compression has emerged as a research direction to address the first challenge, and it has proven to moderate CNN computational load successfully. For the second challenge, various evolutionary algorithms have been proposed thus far. The algorithm proposed in this paper can be viewed as a solution to both challenges. Instead of using constant predefined criteria to evaluate the filters of CNN layers, the proposed algorithm establishes evaluation criteria in online manner during network training based on the combination of each filter’s profit in its layer and the next layer. In addition, the novel method suggested that it inserts new filters into the CNN layers. The proposed algorithm is not simply a pruning strategy but determines the optimal number of filters. Training on multiple CNN architectures allows us to demonstrate the efficacy of our approach empirically. Compared to current pruning algorithms, our algorithm yields a network with a remarkable prune ratio and accuracy. Despite the relatively high computational cost of an epoch in the proposed algorithm in pruning, altogether it achieves the resultant network faster than other algorithms.

Keywords

• convolutional neural network (cnn)
• adaptive architecture
• pruning
• compression