Convolution Smooth: A Post-Training Quantization Method for Convolutional Neural Networks

Abstract

Convolutional neural network (CNN) quantization is an efficient model compression technique primarily used for accelerating inference and optimizing resources. However, existing methods often apply different quantization strategies to activations and weights, without considering their interplay. To address this issue, this paper proposes a new method called Convolution Smooth, which aims to balance the quantization difficulty of activations and weights. This method effectively mitigates the significant accuracy drop typically observed in activation quantization with traditional methods. By appropriately scaling the tensors, the method shifts quantization complexity from activations to weights, ensuring a reasonable distribution of quantization difficulty between activations and weights. Experimental results show that the proposed method is applicable to a wide range of network models and can be seamlessly integrated into multiple post-training quantization (PTQ) methods. Through tensor scaling and the fusion of factors, the network achieves significant accuracy improvements in most cases, particularly when there is a significant discrepancy between the activation values and weight quantization bit-widths. This study provides a new theoretical foundation and technical support for CNN quantization compression.

Keywords

• Convolutional neural network
• post-training quantization
• tensor scaling
• factor fusion