Improving Model Robustness With Frequency Component Modification and Mixing

Abstract

Deep neural networks are sensitive to distribution shifts, such as common corruption and adversarial examples, which occur across various frequency spectra. Numerous studies have been conducted to improve model robustness in the frequency domain. However, research that simultaneously addresses safety metrics, such as corruption, adversaries, consistency, and calibration, is lacking. This study proposes methods named Frequency Component Modification (FCM) and Frequency Component Modification and Mixing (FCMM) to address this gap, as data augmentation techniques in convolutional neural networks (CNNs) for image data. Drawing inspiration from the human vision system’s robustness and frequency spectrum distribution, FCM is successfully developed to introduce larger perturbations to high-frequency components while applying smaller variations to low-frequency components. In addition, FCMM mixes the images generated by FCM, providing diversity at low frequencies and improving the model’s robustness against low-frequency changes. Experimental results demonstrate that FCMM achieves a 1.4%p reduction in mean corruption error (mCE) on both CIFAR-10-C and CIFAR-100-C, and 1.3%p on ImageNet-C compared to PixMix, which uses additional data. Moreover, when combined with DeepAugment (DA), the leading state-of-the-art (SOTA) method in corruption robustness that requires additional networks for augmentations, FCMM achieves an mCE of 57.2% on ImageNet-C, improving upon DA’s standalone performance of 62.0% mCE. These results demonstrate FCMM’s superior performance in corruption robustness, while also showing improvements in other safety metrics, such as adversarial robustness, consistency, and calibration.

Keywords

• Robustness
• image augmentation
• discrete Fourier transforms
• convolutional neural networks
• image classification