Compressing Convolutional Neural Networks by Pruning Density Peak Filters

Abstract

With the recent development of GPUs, the depth of convolutional neural networks (CNNs) has increased, and its structure has become complex. Hence, it is challenging to deploy them into a hardware device owing to its immense computational cost and memory for storage parameters. We propose a method of pruning a filter located near the density peak, which grasps the density of the filter space for each layer to overcome this problem. The density is calculated in the filter space based on the number of neighboring filters within a certain distance around the filter and the distance to a denser space. Moreover, we do not remove all filters at once, but use a method of pruning a certain number iteratively, so that filters can be evenly pruned in multiple locations with high density inside the filter space. After that, we fine-tune the pruned network to restore their performance. The experimental results show the effectiveness of the proposed method with respect to the other methods using CIFAR-10, and ImageNet dataset on VGGNet and ResNet architecture. Notably, on CIFAR-10, our method reduces 60.8% of FLOPs on ResNet56 with 0.31% validation accuracy improvement. Moreover, we achieve up to 51.9% FLOPs reduction with a little accuracy drop on ImageNet for ResNet34.

Keywords

• Convolutional neural networks
• compressing CNNs
• filter pruning
• density peak