Segmented 3D Lung Cube Dataset and Dual-Model Framework for COVID-19 Severity Prediction

Abstract

This research presents two key contributions aimed at improving COVID-19 severity prediction, specifically intubation or death within one month using 3D CT scan data. First, we introduce a novel dataset of 2,000 segmented 3D lung cubes meticulously curated from the STOIC dataset through a robust 10-step preprocessing and segmentation pipeline. Second, we propose two distinct methods for predicting COVID-19 severity. The first method employs a 3D-CNN pretrained on the MosMedData dataset, later fine-tuned on the STOIC dataset with two input layers: one for 3D lung images and another for age and gender metadata. Unlike 2D CNNs, 3D CNNs capture inter-slice information within 3D volumetric images, while Vision Transformers enhance broader context comprehension and capture complex structures more effectively through self-attention mechanisms. Therefore second method known as 3D-EffiBOT leverages a combination of 3D EfficientNetV2 and iBOT architectures to capture both 3D as well as 2D spatial features from volumetric CT scans. 3D EfficientNetV2 with weights obtained after inflating 2D ImageNet weights, was fine-tuned on the STOIC dataset using a dynamic layer unfreezing strategy, while iBOT was employed to extract 2D slice-level features from axial CT slices. Both models were trained using five augmentation techniques and evaluated using stratified 5-fold sampling to address class imbalance, achieving mean AUC score of 0.7862 and 0.7414 for 3D-EffiBOT and 3D-CNN respectively. This study demonstrates the effectiveness of hybrid architectures in medical imaging, yielding substantial improvements over conventional methods. Our findings indicate that hybrid models significantly enhance diagnostic accuracy, presenting a valuable tool for predicting severe COVID-19 outcomes.

Keywords

• 3D CT scans
• COVID-19 severity prediction
• class imbalance
• deep learning
• dynamic augmentation
• EfficientNetV2