Pruning convolution neural networks using filter clustering based on normalized cross-correlation similarity

Abstract

Despite all the recent development and success of deep neural networks, deployment of a deep model onto the resource-constrained devices still remains challenging. However, model pruning can resolve this issue for Convolutional Neural Networks (CNNs), since it is one of the most popular approaches to reducing computational complexities. Therefore, this article presents a pruning model for convolutional neural networks. The proposed method classifies and arranges similar filters into the same cluster where the similarity is calculated using a three-dimensional normalized cross-correlation. Moreover, these steps can be completed entirely based on the filter values while not requiring a set of test images as well as the acquisition of any filter activation. In the research, the performances of the proposed model pruning method have been evaluated, where it is observed that the proposed approach is computationally light and requires significantly less time and resources compared to ML and activation-based approaches. In the experiments, using the VGG16 model on the Cifar10 dataset, the proposed approach results in the pruned model(s) which are comparable in performance with models found using activation-based methods and expensive ML-based methods. Similar results are found when pruning a custom CNN on the MNIST and Fashion MNIST datasets as well.

Keywords

• Convolutional neural networks
• deep neural networks
• pruning