A 3-D Convolutional Vision Transformer for PolSAR Image Classification and Change Detection

Abstract

The scattering properties of targets in polarimetric synthetic aperture radar (PolSAR) images are directly influenced by the targets' orientations, as the scattering properties from the same target with different orientations can be very different. This interpretation diversity caused by the target orientations is one of the primary technical bottlenecks in PolSAR image interpretation. In this article, a 3-D convolutional vision transformer (3-D-Conv-ViT) is proposed to describe the relationship between polarimetric coherent matrices with different polarization orientation angles (POAs) for PolSAR image classification and change detection. First, 3-D convolutional neural networks are used to capture the high-level feature representations of the polarimetric coherent matrix sequence. Second, a new Rotation-3-D-ViT block is proposed to learn the local and global representations of the high-level feature maps. The self-attention mechanism in the ViT can express the regularity of polarimetric coherent matrices with different POAs and improve the PolSAR image interpretation performance. Third, combined with different classifiers, the proposed 3-D-Conv-ViT can be applied to both PolSAR image classification and change detection. Experiments on real PolSAR image datasets demonstrate that the proposed method can overcome the problem of the interpretation ambiguity caused by the target orientation. The classification accuracies of the proposed method can reach 94.01%–99.48%, and the change detection accuracies can reach 93.84%–96.86%.

Keywords

• 3-D convolutional
• change detection
• classification
• polarimetric synthetic aperture radar (PolSAR) image
• vision transformer (ViT)