Magnetic resonance-driven pseudo CT image using patch-based multi-modal feature extraction and ensemble learning with stacked generalisation

Abstract

In recent years, research has been showing a great interest in using magnetic resonance imaging (MRI) as the only modality for radiation therapy (RT) for its superior soft-tissue visualisation and non-ionizing proprieties. Furthermore, MRI-only RT would be of great benefit to eliminate image registration errors, reduce cost and workload. In addition, machine-learning algorithms have been taking the lead in many fields. For instance, in MRI-only RT, machine learning is showing a notable performance compared to other methods owed to the flexibility of these methods towards data regardless of model complexity. In this paper, we present an ensemble learning approach with stacked generalisation to simulate a CT scan from multi-modal MR images from which patch-based shape, texture and spatial features were considered. Feature extraction, fusion and reduction were performed to get the most descriptive and informative features. The ensemble learning model was constructed with two levels of learning were the basic level consisted of three base learners namely: artificial neural networks (ANN), random forests (RF) and k-nearest neighbours (kNN) and the second level representing the stacking learner that takes predictions from the base learner and generates the final predictions. Multiple linear regression (MLR) was used for the stacked generalisation. The proposed ensemble learning with stacked generalisation (ES) approach produced an average mean absolute error (MAE) of 87.60 ± 19.70 and an average mean error (ME) of −4.68 ± 16.43 outperforming the RF method, which produced an average MAE of 106.88 ± 33.20 and an average ME of −5.38 ± 20.77. In addition, average Pearson correlation was 0.92 for the proposed approach compared to 0.89 for RF. Evaluation of the proposed approach shows that stacked generalisation can greatly improve prediction accuracy and reduce bias in electron density estimation.

Keywords

• Multi-modal
• Stacking
• Ensemble learning
• Pseudo-CT
• MRI-only radiation therapy
• Feature fusion