Multibranch Separable 3-D Convolutional Neural Network for Hyperspectral Image Denoising

Abstract

3-D convolutional neural networks (CNNs) offer a great potential spatial–spectral representation for hyperspectral image (HSI), and has achieved promising HSI denoising performance. However, the current 3-D CNNs still suffer from limited nonuniform multicomponent spatial–spectral features encoding, and incur a high computation burden. To address these issues, we draw inspiration from the success of multibranch and separable convolution perspectives, and propose a plug-and-play multibranch separable 3-D convolution block (MS3CB) with different spatial–spectral receptive fields. Specifically, MS3CB comprises mixed regular and nonregular separable 3-D branches, in which the nonregular separable 3-D branch attempts to provide a meaningful nonuniform spatial–spectral features extractor. The separable 3-D convolution in MS3CB factorizes the standard 3-D convolution into a 2-D spatial convolution and a 1-D spectral convolution, which not only reduces model size but also decouples spatial and spectral features in HSI for more flexible spatial–spectral representation. Based on MS3CB, we develop a novel 3-D U-Net for HSI denoising, called HSDU-Net, which uses MS3CB as the basic building block instead of a standard 3-D convolution block. We empirically verify that our separable 3-D convolution block reduces about 64.6% parameters and achieves a certain performance gain over standard 3-D convolution. Extensive experiments further demonstrate that HSDU-Net surpasses several latest baselines on various synthetic and real noisy HSI datasets.

Keywords

• 3-D U-net
• hyperspectral image (HSI) denoising
• separable 3-D convolution
• spatial–spectral representation