Evaluating the Potential of Wavelet Pooling on Improving the Data Efficiency of Light-Weight CNNs

Abstract

Wavelet pooling (WP) in neural network architectures has recently demonstrated more discriminative power than traditional pooling methods. This is mainly because the latter suffer from spatial information loss while wavelet pooling harnesses the power of spectral information. However, the potential of WP in increasing the data efficiency and the extent of this potential have not been investigated yet. Data efficiency refers to the volume of training data required to attain a certain performance level during inference, e.g., recognition accuracy. In this research, we are concerned with evaluating the data efficiency of WP in light-weight architectures–MobileNets. Across a wide variety of seven datasets/applications including object recognition (CIFAR-10, STL-10, CINIC-10, and Intel Image Classification datasets) and diagnostic imaging (colon diseases, brain tumors, and malaria cell images datasets), and while considering classification accuracy as a performance metric, we show that WP achieves an average data saving that exceeds 30% compared to traditional pooling techniques. For other performance measures, namely, precision, recall, and F1-score, we report an average of 30% data saving for object recognition datasets and 22% saving for diagnostic imaging datasets. By focusing on a light-weight architecture, this research further emphasizes the significance of wavelet pooling in training and testing resources-challenged settings such as the applications of edge computing and green deep learning.

Keywords

• CNNs
• classification
• MobileNet
• pooling
• wavelet
• spectral information