A hybrid network integrating convolution and transformer for thymoma segmentation

Abstract

Background: Manual segmentation of thymoma is an onerous, labor-intensive, and subjective task for radiologists. Accordingly, the development of an automatic and efficient method for thymoma segmentation can be valuable for the early detection and diagnosis of this malignancy. Methods: Three hundred and ten subjects were enrolled in this retrospective study and all underwent CECT scans. All the scans were manually labeled by four experienced radiologists. The successful application of convolution neural networks (CNNs) and Transformer in computer vision led us to propose a hybrid CNN–Transformer architecture, named transformer attention Net (TA-Net), that would allow the utilization of both local information from CNN features and the global information encoded by Transformers. U-Net was used as the basic structure and Transformers were inserted into convolution blocks in the encoder. In addition, attention gates were embedded in skip connections to highlight salient features. Comparison of the accuracy, intersection over Union (IoU), Dice score, and Boundary F1 contour matching score (BFScore) between the predicted segmentation and the manual labels were utilized to evaluate segmentation performance. Results: For thymoma segmentation using TA-Net, the accuracy, Dice score, IoU, and BFScore were 92.49%, 89.92%, 83.80%, and 0.8945, respectively, and no significant differences were detected among tumor types and enhanced phases. Our proposed method achieved the best performance when compared with state-of-the-art methods. Conclusion: The proposed method, which combines CNNs with Transformer, achives outstanding performance in thymoma segmentation compared with previous methods. TA-Net may provide consistent and reproducible delineation, thereby assisting radiologists in clinical applications.

Keywords

• Semantic segmentation
• Convolution neural network
• Transformer
• Attention mechanism
• Thymoma