BMC Medical Research Methodology (Feb 2020)
Framework for personalized prediction of treatment response in relapsing remitting multiple sclerosis
Abstract
Abstract Background Personalized healthcare promises to successfully advance the treatment of heterogeneous neurological disorders such as relapsing remitting multiple sclerosis by addressing the caveats of traditional healthcare. This study presents a framework for personalized prediction of treatment response based on real-world data from the NeuroTransData network. Methods A framework for personalized prediction of response to various treatments currently available for relapsing remitting multiple sclerosis patients was proposed. Two indicators of therapy effectiveness were used: number of relapses, and confirmed disability progression. The following steps were performed: (1) Data preprocessing and selection of predictors according to quality and inclusion criteria; (2) Implementation of hierarchical Bayesian generalized linear models for estimating treatment response; (3) Validation of the resulting predictive models based on several performance measures and routines, together with additional analyses that focus on evaluating the usability in clinical practice, such as comparing predicted treatment response with the empirically observed course of multiple sclerosis for different adherence profiles. Results The results revealed that the predictive models provide robust and accurate predictions and generalize to new patients and clinical sites. Three different out-of-sample validation schemes (10-fold cross-validation, leave-one-site-out cross-validation, and excluding a test set) were employed to assess generalizability based on three different statistical performance measures (mean squared error, Harrell’s concordance statistic, and negative log-likelihood). Sensitivity to different choices of the priors, to the characteristics of the underlying patient population, and to the sample size, was assessed. Finally, it was shown that model predictions are clinically meaningful. Conclusions Applying personalized predictive models in relapsing remitting multiple sclerosis patients is still new territory that is rapidly evolving and has many challenges. The proposed framework addresses the following challenges: robustness and accuracy of the predictions, generalizability to new patients and clinical sites and comparability of the predicted effectiveness of different therapies. The methodological and clinical soundness of the results builds the basis for a future support of patients and doctors when the current treatment is not generating the desired effect and they are considering a therapy switch. Graphical abstract (A) The framework is developed using quality-proven real-world data of patients with relapsing remitting multiple sclerosis. Patients have heterogeneous individual characteristics and diverse disease profiles, indicated for example by variations in frequency of relapses and degree of disability. Longitudinal characteristics regarding disease history (e.g. number of previous relapses in the last 12 months) are extracted at the time of an intended therapy switch, i.e. at time point “Today” (left). All clinical parameters are captured in a standardized way (right). (B) The model predicts the course of the disease based on the observed data (panel A), and is able to account for the impact of various available therapies on chosen clinical endpoints. The resulting ranking of therapies has a dependency on patient characteristics, illustrated here by a different highest ranked therapy depending on the number of relapse in the previous 12 months. (C) The model is evaluated for various generalization properties. Compared to performance on the training set (gray) it is able to predict for new patients not part of the training set (red).Top: Prediction for new patients. Middle: Prediction for new clinical sites. Bottom: Prediction for different time windows. (D) In order to assess the clinical impact of the model, disease activity is compared between patients treated with the highest ranked therapy and those treated with any of the other therapies. Patients adhering to the highest ranked therapy are associated with a better disease outcome when compared to those who did not.
Keywords
