Identification of ulcerative colitis and Crohn's disease based on spatial and bilinear attention network

Abstract

Objective To identify ulcerative colitis (UC) and Crohn's disease (CD) with aid of deep learning technology for endoscopists. Methods From January 2018 to November 2020, the endoscopic images of 1 576 subjects (including 34 300 CD, UC and normal images) were collected from the Department of Gastroenterology of Army Medical Center of PLA and Sir Run Run Shaw Hospital.The training set and test set were randomly divided according to the ratio of 9:1 to train and test the neural network.A novel spatial and bilinear deep network (SABA-ResNet) was constructed on the basis of ResNet50.The spatial attention mechanism was introduced, and the receptive field was expanded by dilated convolution to leverage contextual information, which was combined with the local induction of standard convolution to adaptively focus the lesion region.Bilinear attention was applied to improve the feature representation ability of the network, and the second-order information was used to weight the channel information of the feature map, so as to improve the classification performance of the model. Results The overall accuracy of SABA-ResNet for the recognition of CD, UC and normal tissues on the test set was 92.67%(95%CI: 91.91~93.37), the AUC value was 0.978(95%CI: 0.972~0.983), 0.977(95%CI: 0.971~0.982) and 0.999(95%CI: 0.998~1.000), the sensitivity was 88.40%, 89.07% and 98.89%, the specificity was 95.49%, 94.88% and 98.93%, and the F1 value was 88.80%, 89.01% and 98.60%, respectively.The ablation experiment and the class activation map suggested that spatial attention and bilinear attention could help the model capture more features of the lesion region. Conclusion Our constructed network combines spatial attention and bilinear attention, achieves excellent performance in the recognition of CD, UC and normal tissue, and effectively assist endoscopists in the diagnosis of UC and CD.

Keywords

• inflammatory bowel disease
• deep learning
• ulcerative colitis
• crohn's disease