NeuroImage: Clinical (Jan 2021)
Development and evaluation of a manual segmentation protocol for deep grey matter in multiple sclerosis: Towards accelerated semi-automated references
- Alexandra de Sitter,
- Jessica Burggraaff,
- Fabian Bartel,
- Miklos Palotai,
- Yaou Liu,
- Jorge Simoes,
- Serena Ruggieri,
- Katharina Schregel,
- Stefan Ropele,
- Maria A. Rocca,
- Claudio Gasperini,
- Antonio Gallo,
- Menno M. Schoonheim,
- Michael Amann,
- Marios Yiannakas,
- Deborah Pareto,
- Mike P. Wattjes,
- Jaume Sastre-Garriga,
- Ludwig Kappos,
- Massimo Filippi,
- Christian Enzinger,
- Jette Frederiksen,
- Bernard Uitdehaag,
- Charles R.G. Guttmann,
- Frederik Barkhof,
- Hugo Vrenken
Affiliations
- Alexandra de Sitter
- Radiology and Nuclear Medicine, MS Center Amsterdam, Amsterdam Neuroscience, Amsterdam UMC, Location VUmc, Amsterdam, NL, Netherlands
- Jessica Burggraaff
- Department of Neurology, MS Center Amsterdam, Amsterdam Neuroscience, Amsterdam UMC, Location VUmc, Amsterdam, NL, Netherlands; Corresponding author at: Amsterdam UMC, location VUmc, Department of Neurology, PO Box 7057, 1007 MB Amsterdam, The Netherlands.
- Fabian Bartel
- Radiology and Nuclear Medicine, MS Center Amsterdam, Amsterdam Neuroscience, Amsterdam UMC, Location VUmc, Amsterdam, NL, Netherlands
- Miklos Palotai
- Center for Neurological Imaging, Department of radiology, Brigham and Women’s Hospital, Harvard Medical School Boston, MA, USA
- Yaou Liu
- Radiology and Nuclear Medicine, MS Center Amsterdam, Amsterdam Neuroscience, Amsterdam UMC, Location VUmc, Amsterdam, NL, Netherlands
- Jorge Simoes
- Radiology and Nuclear Medicine, MS Center Amsterdam, Amsterdam Neuroscience, Amsterdam UMC, Location VUmc, Amsterdam, NL, Netherlands
- Serena Ruggieri
- Department of Human Neurosciences, “Sapienza” University of Rome, Rome, IT, Italy; Department of Neurosciences, San Camillo Forlanini Hospital, Rome, IT, Italy
- Katharina Schregel
- Center for Neurological Imaging, Department of radiology, Brigham and Women’s Hospital, Harvard Medical School Boston, MA, USA; Institute of Neuroradiology, University Medical Center Goettingen, Goettingen, DE, Germany
- Stefan Ropele
- Department of Neurology, Medical University of Graz, Graz, AT, Austria
- Maria A. Rocca
- Neuroimaging Research Unit, Institute of Experimental Neurology, Division of Neuroscience, United States; Neurology Unit, San Raffaele Scientific Institute, UniSR, Milan, IT, Italy
- Claudio Gasperini
- Department of Neurosciences, San Camillo Forlanini Hospital, Rome, IT, Italy
- Antonio Gallo
- Division of Neurology and 3T MRI Research Center, Department of Advanced Medical and Surgical Sciences, University of Campania “Luigi Vanvitelli”, Naples, IT, Italy
- Menno M. Schoonheim
- Department of Anatomy and Neurosciences, MS Center Amsterdam, Amsterdam Neuroscience, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam, NL, Netherlands
- Michael Amann
- Medical Image Analysis Center (MIAC), United States; Neurologic Clinic and Policlinic and Neuroradiology, Department of Biomedical Engineering, University Hospital Basel, Basel, CH, Switzerland
- Marios Yiannakas
- Department of Neuroinflammation, Institute of Neurology, UCL, London, UK
- Deborah Pareto
- Section of Neuroradiology and MRI Unit, Department of Radiology, University Hospital Valld’Hebron, Autonomous University of Barcelona, Barcelona, ES, Spain
- Mike P. Wattjes
- Radiology and Nuclear Medicine, MS Center Amsterdam, Amsterdam Neuroscience, Amsterdam UMC, Location VUmc, Amsterdam, NL, Netherlands; Deptartment of Diagnostic and Interventional Neuroradiology, Hannover Medical School, Hannover, DE, Germany
- Jaume Sastre-Garriga
- Department of Neurology, University Hospital iValld’Hebron, Autonomous University of Barcelona, Barcelona, ES, Spain
- Ludwig Kappos
- Neurologic Clinic and Policlinic and Neuroradiology, Department of Biomedical Engineering, University Hospital Basel, Basel, CH, Switzerland
- Massimo Filippi
- Neuroimaging Research Unit, Institute of Experimental Neurology, Division of Neuroscience, United States; Neurology Unit, San Raffaele Scientific Institute, UniSR, Milan, IT, Italy; Neurophysiology Unit, San Raffaele Scientific Institute, Italy; Vita-Salute San Raffaele University, Milan, IT, Italy
- Christian Enzinger
- Division of Neuroradiology, Vascular and Interventional Radiology, Department of Radiology, Medical University of Graz, Graz, AT, Austria
- Jette Frederiksen
- Department of Neurology, Glostrup University Hospital, Copenhagen, DK, Denmark
- Bernard Uitdehaag
- Department of Neurology, MS Center Amsterdam, Amsterdam Neuroscience, Amsterdam UMC, Location VUmc, Amsterdam, NL, Netherlands
- Charles R.G. Guttmann
- Center for Neurological Imaging, Department of radiology, Brigham and Women’s Hospital, Harvard Medical School Boston, MA, USA
- Frederik Barkhof
- Radiology and Nuclear Medicine, MS Center Amsterdam, Amsterdam Neuroscience, Amsterdam UMC, Location VUmc, Amsterdam, NL, Netherlands; Institutes of Neurology & Healthcare Engineering, UCL, London, UK
- Hugo Vrenken
- Radiology and Nuclear Medicine, MS Center Amsterdam, Amsterdam Neuroscience, Amsterdam UMC, Location VUmc, Amsterdam, NL, Netherlands
- Journal volume & issue
-
Vol. 30
p. 102659
Abstract
Background: Deep grey matter (dGM) structures, particularly the thalamus, are clinically relevant in multiple sclerosis (MS). However, segmentation of dGM in MS is challenging; labeled MS-specific reference sets are needed for objective evaluation and training of new methods. Objectives: This study aimed to (i) create a standardized protocol for manual delineations of dGM; (ii) evaluate the reliability of the protocol with multiple raters; and (iii) evaluate the accuracy of a fast-semi-automated segmentation approach (FASTSURF). Methods: A standardized manual segmentation protocol for caudate nucleus, putamen, and thalamus was created, and applied by three raters on multi-center 3D T1-weighted MRI scans of 23 MS patients and 12 controls. Intra- and inter-rater agreement was assessed through intra-class correlation coefficient (ICC); spatial overlap through Jaccard Index (JI) and generalized conformity index (CIgen). From sparse delineations, FASTSURF reconstructed full segmentations; accuracy was assessed both volumetrically and spatially. Results: All structures showed excellent agreement on expert manual outlines: intra-rater JI > 0.83; inter-rater ICC ≥ 0.76 and CIgen ≥ 0.74. FASTSURF reproduced manual references excellently, with ICC ≥ 0.97 and JI ≥ 0.92. Conclusions: The manual dGM segmentation protocol showed excellent reproducibility within and between raters. Moreover, combined with FASTSURF a reliable reference set of dGM segmentations can be produced with lower workload.
