TrUNet: Dual-Branch Network by Fusing CNN and Transformer for Skin Lesion Segmentation

Abstract

In the medical field, precise segmentation of skin lesion areas is essential for accurate diagnosis and treatment of diseases. Due to the varied morphologies and fuzzy boundaries of skin lesions, as well as interference from hair coverage, segmentation tasks are extremely challenging. To address the problem, a network called TrUNet is proposed, which combines the advantages of Transformer and convolutional neural networks (CNNs). Transformer and Res2Net are taken as two branches of the encoder in this network, with the goal of extracting rich global information for precise lesion segmentation in medical images. Firstly, the TrFusion module was designed to selectively fuse complementary features extracted by the Transformer branch and the Res2Net branch in the encoder, enhancing important information while suppressing irrelevant details. Secondly, the Multi-Scale Feature Aggregation (MFA) module was designed to fuse feature representations from different stages of the same branch to complement positional and spatial information. Finally, to validate the effectiveness of the proposed method, experiments were conducted on the ISIC2017, ISIC2018, and PH2 datasets. TrUNet achieved Dice coefficient of 90.61%, IoU of 84.25%, and Accuracy of 94.74% on the ISIC2018 dataset. This indicates that our model has enormous potential in the field of medical image segmentation.

Keywords

• Deep learning
• convolutional neural network
• transformer
• skin lesion segmentation