NeuroImage (Jul 2024)
Lack of evidence for predictive utility from resting state fMRI data for individual exposure-based cognitive behavioral therapy outcomes: A machine learning study in two large multi-site samples in anxiety disorders
- Kevin Hilbert,
- Joscha Böhnlein,
- Charlotte Meinke,
- Alice V. Chavanne,
- Till Langhammer,
- Lara Stumpe,
- Nils Winter,
- Ramona Leenings,
- Dirk Adolph,
- Volker Arolt,
- Sophie Bischoff,
- Jan C. Cwik,
- Jürgen Deckert,
- Katharina Domschke,
- Thomas Fydrich,
- Bettina Gathmann,
- Alfons O. Hamm,
- Ingmar Heinig,
- Martin J. Herrmann,
- Maike Hollandt,
- Jürgen Hoyer,
- Markus Junghöfer,
- Tilo Kircher,
- Katja Koelkebeck,
- Martin Lotze,
- Jürgen Margraf,
- Jennifer L.M. Mumm,
- Peter Neudeck,
- Paul Pauli,
- Andre Pittig,
- Jens Plag,
- Jan Richter,
- Isabelle C. Ridderbusch,
- Winfried Rief,
- Silvia Schneider,
- Hanna Schwarzmeier,
- Fabian R. Seeger,
- Niklas Siminski,
- Benjamin Straube,
- Thomas Straube,
- Andreas Ströhle,
- Hans-Ulrich Wittchen,
- Adrian Wroblewski,
- Yunbo Yang,
- Kati Roesmann,
- Elisabeth J. Leehr,
- Udo Dannlowski,
- Ulrike Lueken
Affiliations
- Kevin Hilbert
- Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany; Department of Psychology, HMU Health and Medical University Erfurt, Erfurt, Germany
- Joscha Böhnlein
- Institute for Translational Psychiatry, University of Münster, Germany; Corresponding author at: University of Münster, Institute for Translational Psychiatry, Albert-Schweitzer-Campus 1, Building A9a, 48147, Münster.
- Charlotte Meinke
- Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany
- Alice V. Chavanne
- Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany; Université Paris-Saclay, INSERM U1299 “Trajectoires développementales et psychiatrie”, CNRS UMR 9010 Centre Borelli, Ecole Normale Supérieure Paris-Saclay, France
- Till Langhammer
- Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany
- Lara Stumpe
- Institute for Translational Psychiatry, University of Münster, Germany
- Nils Winter
- Institute for Translational Psychiatry, University of Münster, Germany
- Ramona Leenings
- Institute for Translational Psychiatry, University of Münster, Germany
- Dirk Adolph
- Mental Health Research and Treatment Center, Faculty of Psychology, Ruhr-Universität Bochum, Bochum, Germany
- Volker Arolt
- Institute for Translational Psychiatry, University of Münster, Germany
- Sophie Bischoff
- Department of Psychiatry and Psychotherapy, Campus Charité Mitte, Charité ‐ Universitätsmedizin Berlin, Berlin, Germany
- Jan C. Cwik
- Department of Clinical Psychology and Psychotherapy, Faculty of Human Sciences, Universität zu Köln, Germany
- Jürgen Deckert
- Center for Mental Health, Department of Psychiatry, Psychosomatics, and Psychotherapy, University Hospital of Würzburg, Germany
- Katharina Domschke
- Department of Psychiatry and Psychotherapy, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany
- Thomas Fydrich
- Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany
- Bettina Gathmann
- Institute of Medical Psychology and Systems Neuroscience, University of Münster, Germany
- Alfons O. Hamm
- Department of Biological and Clinical Psychology, University of Greifswald, Greifswald, Germany
- Ingmar Heinig
- Institute of Clinical Psychology & Psychotherapy, Technische Universität Dresden, Dresden, Germany
- Martin J. Herrmann
- Center for Mental Health, Department of Psychiatry, Psychosomatics, and Psychotherapy, University Hospital of Würzburg, Germany
- Maike Hollandt
- Department of Biological and Clinical Psychology, University of Greifswald, Greifswald, Germany
- Jürgen Hoyer
- Institute of Clinical Psychology & Psychotherapy, Technische Universität Dresden, Dresden, Germany
- Markus Junghöfer
- Institute for Biomagnetism and Biosignalanalysis, University of Münster, Germany
- Tilo Kircher
- Department of Psychiatry and Psychotherapy, University of Marburg, Marburg, Germany
- Katja Koelkebeck
- LVR-University-Hospital Essen, Department of Psychiatry and Psychotherapy, University of Duisburg-Essen, Essen, Germany
- Martin Lotze
- Functional Imaging Unit. Diagnostic Radiology and Neuroradiology, University Medicine Greifswald, Greifswald, Germany
- Jürgen Margraf
- Mental Health Research and Treatment Center, Faculty of Psychology, Ruhr-Universität Bochum, Bochum, Germany
- Jennifer L.M. Mumm
- Department of Psychiatry and Psychotherapy, Campus Charité Mitte, Charité ‐ Universitätsmedizin Berlin, Berlin, Germany
- Peter Neudeck
- Protect‐AD Study Site Cologne, Cologne, Germany
- Paul Pauli
- Department of Psychology, University of Würzburg, Würzburg, Germany
- Andre Pittig
- Translational Psychotherapy, Institute of Psychology, University of Göttingen, Germany
- Jens Plag
- Department of Psychiatry and Psychotherapy, Campus Charité Mitte, Charité ‐ Universitätsmedizin Berlin, Berlin, Germany; Department of Psychiatry, Psychotherapy and Psychosomatics, Alexianer Krankenhaus Hedwigshoehe, St. Hedwig Kliniken, Berlin, Germany
- Jan Richter
- Department of Biological and Clinical Psychology, University of Greifswald, Greifswald, Germany; Department of Experimental Psychopathology, University of Hildesheim, Hildesheim, Germany
- Isabelle C. Ridderbusch
- Department of Psychiatry and Psychotherapy, University of Marburg, Marburg, Germany
- Winfried Rief
- Department of Clinical Psychology and Psychotherapy, Faculty of Psychology & Center for Mind, Brain and Behavior – CMBB, Philipps‐University of Marburg, Marburg, Germany
- Silvia Schneider
- Faculty of Psychology, Clinical Child and Adolescent Psychology, Mental Health Research and Treatment Center, Ruhr-Universität Bochum, Bochum, Germany
- Hanna Schwarzmeier
- Center for Mental Health, Department of Psychiatry, Psychosomatics, and Psychotherapy, University Hospital of Würzburg, Germany
- Fabian R. Seeger
- Center for Mental Health, Department of Psychiatry, Psychosomatics, and Psychotherapy, University Hospital of Würzburg, Germany
- Niklas Siminski
- Center for Mental Health, Department of Psychiatry, Psychosomatics, and Psychotherapy, University Hospital of Würzburg, Germany
- Benjamin Straube
- Department of Psychiatry and Psychotherapy, University of Marburg, Marburg, Germany
- Thomas Straube
- Institute of Psychology, Unit of Clinical Psychology and Psychotherapy in Childhood and Adolescence, University of Osnabrueck, Osnabruck, Germany
- Andreas Ströhle
- Department of Psychiatry and Psychotherapy, Campus Charité Mitte, Charité ‐ Universitätsmedizin Berlin, Berlin, Germany
- Hans-Ulrich Wittchen
- Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany; Department of Psychology, HMU Health and Medical University Erfurt, Erfurt, Germany; Institute for Translational Psychiatry, University of Münster, Germany; Université Paris-Saclay, INSERM U1299 “Trajectoires développementales et psychiatrie”, CNRS UMR 9010 Centre Borelli, Ecole Normale Supérieure Paris-Saclay, France; Mental Health Research and Treatment Center, Faculty of Psychology, Ruhr-Universität Bochum, Bochum, Germany; Department of Psychiatry and Psychotherapy, Campus Charité Mitte, Charité ‐ Universitätsmedizin Berlin, Berlin, Germany; Department of Clinical Psychology and Psychotherapy, Faculty of Human Sciences, Universität zu Köln, Germany; Center for Mental Health, Department of Psychiatry, Psychosomatics, and Psychotherapy, University Hospital of Würzburg, Germany; Department of Psychiatry and Psychotherapy, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany; Institute of Medical Psychology and Systems Neuroscience, University of Münster, Germany; Department of Biological and Clinical Psychology, University of Greifswald, Greifswald, Germany; Institute of Clinical Psychology & Psychotherapy, Technische Universität Dresden, Dresden, Germany; Institute for Biomagnetism and Biosignalanalysis, University of Münster, Germany; Department of Psychiatry and Psychotherapy, University of Marburg, Marburg, Germany; LVR-University-Hospital Essen, Department of Psychiatry and Psychotherapy, University of Duisburg-Essen, Essen, Germany; Functional Imaging Unit. Diagnostic Radiology and Neuroradiology, University Medicine Greifswald, Greifswald, Germany; Protect‐AD Study Site Cologne, Cologne, Germany; Institut für Klinische Psychologie und Psychotherapie, TU Chemnitz, Germany; Department of Psychology, University of Würzburg, Würzburg, Germany; Translational Psychotherapy, Institute of Psychology, University of Göttingen, Germany; Department of Psychiatry, Psychotherapy and Psychosomatics, Alexianer Krankenhaus Hedwigshoehe, St. Hedwig Kliniken, Berlin, Germany; Department of Experimental Psychopathology, University of Hildesheim, Hildesheim, Germany; Department of Clinical Psychology and Psychotherapy, Faculty of Psychology & Center for Mind, Brain and Behavior – CMBB, Philipps‐University of Marburg, Marburg, Germany; Faculty of Psychology, Clinical Child and Adolescent Psychology, Mental Health Research and Treatment Center, Ruhr-Universität Bochum, Bochum, Germany; Institute of Psychology, Unit of Clinical Psychology and Psychotherapy in Childhood and Adolescence, University of Osnabrueck, Osnabruck, Germany; Psychiatric University Hospital, Ludwig-Maximilians-University München, Germany; German Center for Mental Health (DZPG), partner site Berlin/Potsdam, Germany
- Adrian Wroblewski
- Department of Psychiatry and Psychotherapy, University of Marburg, Marburg, Germany
- Yunbo Yang
- Department of Psychiatry and Psychotherapy, University of Marburg, Marburg, Germany
- Kati Roesmann
- Institute for Biomagnetism and Biosignalanalysis, University of Münster, Germany; Institute of Psychology, Unit of Clinical Psychology and Psychotherapy in Childhood and Adolescence, University of Osnabrueck, Osnabruck, Germany
- Elisabeth J. Leehr
- Institute for Translational Psychiatry, University of Münster, Germany
- Udo Dannlowski
- Institute for Translational Psychiatry, University of Münster, Germany
- Ulrike Lueken
- Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany; German Center for Mental Health (DZPG), partner site Berlin/Potsdam, Germany
- Journal volume & issue
-
Vol. 295
p. 120639
Abstract
Data-based predictions of individual Cognitive Behavioral Therapy (CBT) treatment response are a fundamental step towards precision medicine. Past studies demonstrated only moderate prediction accuracy (i.e. ability to discriminate between responders and non-responders of a given treatment) when using clinical routine data such as demographic and questionnaire data, while neuroimaging data achieved superior prediction accuracy. However, these studies may be considerably biased due to very limited sample sizes and bias-prone methodology. Adequately powered and cross-validated samples are a prerequisite to evaluate predictive performance and to identify the most promising predictors. We therefore analyzed resting state functional magnet resonance imaging (rs-fMRI) data from two large clinical trials to test whether functional neuroimaging data continues to provide good prediction accuracy in much larger samples. Data came from two distinct German multicenter studies on exposure-based CBT for anxiety disorders, the Protect-AD and SpiderVR studies. We separately and independently preprocessed baseline rs-fMRI data from n = 220 patients (Protect-AD) and n = 190 patients (SpiderVR) and extracted a variety of features, including ROI-to-ROI and edge-functional connectivity, sliding-windows, and graph measures. Including these features in sophisticated machine learning pipelines, we found that predictions of individual outcomes never significantly differed from chance level, even when conducting a range of exploratory post-hoc analyses. Moreover, resting state data never provided prediction accuracy beyond the sociodemographic and clinical data. The analyses were independent of each other in terms of selecting methods to process resting state data for prediction input as well as in the used parameters of the machine learning pipelines, corroborating the external validity of the results. These similar findings in two independent studies, analyzed separately, urge caution regarding the interpretation of promising prediction results based on neuroimaging data from small samples and emphasizes that some of the prediction accuracies from previous studies may result from overestimation due to homogeneous data and weak cross-validation schemes. The promise of resting-state neuroimaging data to play an important role in the prediction of CBT treatment outcomes in patients with anxiety disorders remains yet to be delivered.
