Jisuanji kexue yu tansuo (May 2022)
Review of Winograd Fast Convolution Technique Research
Abstract
Convolutional neural networks (CNN) have been widely used in various fields and have played an impor-tant role. Convolution operator is the basic component of CNN, and it is also the most time-consuming part. In recent years, researchers have proposed several fast convolution algorithms including FFT and Winograd. Among them, Winograd convolution has quickly become the first choice for fast convolution implementation on convolu-tion operators with small convolution kernels, because it greatly reduces the multiplication operations in convolu-tion and occupies less memory. Related work focuses on the generalization, extension and implementation on various architectures of the Winograd convolution, but there are no researchers who have systematically summa-rized the Winograd convolution algorithm. This paper aims to provide detailed reference for follow-up researchers, and summarizes all related work since the introduction of Winograd convolution. Firstly, the introduction of Winograd minimum filtering algorithm and Winograd convolution is described, the generalization and extension of Winograd convolution are introduced, and the detailed differences between existing implementations are also listed. The optimization of Winograd convolution is introduced from the three aspects of sparse pruning, low precision and quantization, and numerical stability, and the advantages and disadvantages of the specific methods are elaborated. The implementations and optimizations of various architectures are classified and summarized, the general optimi-zation methods available for implementation on each platform are compared, and the practical application of Winograd convolution is also introduced. Finally, a brief summary of the content is made, the limitations of existing research are analyzed, and a preliminary outlook for the possible future directions is made.
Keywords
