SAMPE: Auto-Prompting SAM for Generalizable Power Equipment Image Segmentation

Abstract

Power equipment image segmentation is challenging as it involves objects of various scales/sizes, illustration conditions, and imaging angles, making task-specific deep learning approaches struggle to extract scene-invariant features for segmentation and resulting in poor generalization in practice. To address this, how to build generalizable feature extraction is crucial. Inspired by the latest vision foundation model, the Segment Anything Model (SAM), which is trained by super-large-scale data, we propose SAMPE for generalizable power equipment image segmentation. Specifically, SAMPE inherits the outstanding feature extraction capability of SAM with specific designs for feature transfer to power scenarios through adapters. To supplement local information, a CNN image encoder, together with an attention module for information exchange, is introduced to work jointly with the ViT image encoder of SAM. In addition, to overcome the labor-dependent nature of SAM for semi-automatic interaction, SAMPE allows automatic prompting for segmentation. Extensive experiments conducted on two large-scale datasets, including SESD and TTPLA for substation equipment segmentation, demonstrate the superiority of SAMPE outperforming mainstream task-specific deep learning-based segmentation approaches. More importantly, compared to SAM, SAMPE works end-to-end, making it effortlessly applicable to power equipment fault detection, communication operations, maintenance training, etc.

Keywords

• Automatic prompting
• feature adaptation
• power equipment image segmentation
• SAM
• transformer