Fine-Grained Fish Classification From Small to Large Datasets With Vision Transformers

Abstract

Fish species Fine-Grained Visual Classification (FGVC) is important for ecological research, environmental management, and biodiversity monitoring, as accurate fish species identification is crucial for assessing the health of marine ecosystems, monitoring changes in biodiversity, and converting conservation plans into action. Although Convolutional Neural Network (CNN)s have been the conventional approach for FGVC, their effectiveness in differentiating visually similar species is not always satisfactory. The advent of Vision Transformer (ViT)s, in particular the Shifted window (Swin) Transformer, has demonstrated potential in addressing these issues by using sophisticated self-attention and feature extraction techniques. This paper proposes a method of combining the FGVC Plug-in Module (FGVC-PIM) and the Swin Transformer. The FGVC-PIM improves classification by concentrating on the most discriminative image regions, while the Swin Transformer acts as the framework and provides strong hierarchical feature extraction. The performance of the method was assessed on 14 different datasets, which included 19 distinct subsets with varying environmental conditions and image quality. With the proposed method it was achieved state-of-the-art results in 13 of these subsets, exhibiting better accuracy and robustness than previous methods, in 2 subsets (not yet explored by other authors) new baseline results are presented, and in the remaining 4 it was achieved results always above 83%.

Keywords

• Computer vision
• convolutional neural networks
• fine-grained visual classification
• marine biodiversity monitoring
• swin transformer