An Edge-Aware Multitask Network Based on CNN and Transformer Backbone for Farmland Instance Segmentation

Abstract

Accurate farmland instance segmentation is a fundamental task in smart agriculture. Currently, extracting farmland edges as constraint information for farmland segmentation has become the mainstream solution. However, in actual farmland segmentation, single farmland may exhibit multiple cultivation states (bare ground, initial sowing, and mature phase). This may lead to many internal pseudoedges during edge extraction, which is detrimental to farmland segmentation. To address this, we propose a multiscale edge-aware framework based on a hybrid of CNN and transformer (CTMENet) for farmland instance segmentation to mitigate the issues caused by pseudoedges. First, we propose a hybrid backbone of CNN and transformer based on channel gating code, which is used to enhance the holistic recognition of farmland. Then, we propose a multiscale edge-aware head that extracts rich semantic edge features at multiple levels, enhancing the perception of farmland edges. To reduce the pseudoedges, we design an edge loss function that is relatively sensitive to pseudoedges. Finally, we generate high-quality farmland proposals (HFP) based on the edge-aware head to enhance the induction of farmland features, and the HFP is integrated with the mask head through consistency constraints. The experimental results show that the AP metric of our method improves by 4.5% on average compared with the baselines.

Keywords

• Edge extraction
• farmland instance segmentation
• hybrid backbone
• multitask