Robust Fish Recognition Using Foundation Models toward Automatic Fish Resource Management

Abstract

Resource management for fisheries plays a pivotal role in fostering a sustainable fisheries industry. In Japan, resource surveys rely on manual measurements by staff, incurring high costs and limitations on the number of feasible measurements. This study endeavors to revolutionize resource surveys by implementing image-recognition technology. Our methodology involves developing a system that detects individual fish regions in images and automatically identifies crucial keypoints for accurate fish length measurements. We use grounded-segment-anything (Grounded-SAM), a foundation model for fish instance segmentation. Additionally, we employ a Mask Keypoint R-CNN trained on the fish image bank (FIB), which is an original dataset of fish images, to accurately detect significant fish keypoints. Diverse fish images were gathered for evaluation experiments, demonstrating the robust capabilities of the proposed method in accurately detecting both fish regions and keypoints.

Keywords

• resource survey
• foundation models
• Grounded-SAM
• mask keypoint R-CNN
• fish image bank dataset