CNX-B2: A Novel CNN-Transformer Approach For Chest X-Ray Medical Report Generation

Abstract

Medical imaging techniques are the most popular non-invasive methods to diagnose chest diseases. Chest X-ray scans are employed commonly to detect chronic obstructive pulmonary diseases and other respiratory diseases. Despite the significance of these diagnostic methods, the process of disease detection and the subsequent task of CXR report writing is tedious for radiologists. Therefore, Automated radiological report generation is a highly desirable task for radiologists. Previous studies were focused on the automated generation of medical reports to achieve greater quantitative scores rather than focusing on the quality of reports. Such approaches suffer from the problem of generating normal reports for CXR with diseases. Additionally, the absence of clear segregation between normal and abnormal samples in publicly available datasets its impossible to evaluate the performance of models in generating rare abnormal reports. To address these issues, we propose CNX-B2 which is a Convolutional Neural Network (CNN) combined with a Transformer approach to generate medical reports. The proposed encoder is designed to be both hybrid and efficient, capturing meaningful spatial features through inherent convolution biases. This enables the transformer-based decoder to robustly convert these features into coherent medical reports. Secondly, we also introduce a new radiological report dataset to evaluate model performances on abnormal reports separately. Our proposed model is further evaluated on the IU-Xray dataset, achieving competitive scores of 0.479 BLEU-1, 0.188 METEOR, and 0.586 CIDER.

Keywords

• Convolution neural networks
• transformers
• medical report generation
• natural language processing