Nature Communications (Dec 2018)
Why rankings of biomedical image analysis competitions should be interpreted with care
- Lena Maier-Hein,
- Matthias Eisenmann,
- Annika Reinke,
- Sinan Onogur,
- Marko Stankovic,
- Patrick Scholz,
- Tal Arbel,
- Hrvoje Bogunovic,
- Andrew P. Bradley,
- Aaron Carass,
- Carolin Feldmann,
- Alejandro F. Frangi,
- Peter M. Full,
- Bram van Ginneken,
- Allan Hanbury,
- Katrin Honauer,
- Michal Kozubek,
- Bennett A. Landman,
- Keno März,
- Oskar Maier,
- Klaus Maier-Hein,
- Bjoern H. Menze,
- Henning Müller,
- Peter F. Neher,
- Wiro Niessen,
- Nasir Rajpoot,
- Gregory C. Sharp,
- Korsuk Sirinukunwattana,
- Stefanie Speidel,
- Christian Stock,
- Danail Stoyanov,
- Abdel Aziz Taha,
- Fons van der Sommen,
- Ching-Wei Wang,
- Marc-André Weber,
- Guoyan Zheng,
- Pierre Jannin,
- Annette Kopp-Schneider
Affiliations
- Lena Maier-Hein
- Division of Computer Assisted Medical Interventions (CAMI), German Cancer Research Center (DKFZ)
- Matthias Eisenmann
- Division of Computer Assisted Medical Interventions (CAMI), German Cancer Research Center (DKFZ)
- Annika Reinke
- Division of Computer Assisted Medical Interventions (CAMI), German Cancer Research Center (DKFZ)
- Sinan Onogur
- Division of Computer Assisted Medical Interventions (CAMI), German Cancer Research Center (DKFZ)
- Marko Stankovic
- Division of Computer Assisted Medical Interventions (CAMI), German Cancer Research Center (DKFZ)
- Patrick Scholz
- Division of Computer Assisted Medical Interventions (CAMI), German Cancer Research Center (DKFZ)
- Tal Arbel
- Centre for Intelligent Machines, McGill University
- Hrvoje Bogunovic
- Christian Doppler Laboratory for Ophthalmic Image Analysis, Department of Ophthalmology, Medical University Vienna
- Andrew P. Bradley
- Science and Engineering Faculty, Queensland University of Technology
- Aaron Carass
- Department of Electrical and Computer Engineering, Department of Computer Science, Johns Hopkins University
- Carolin Feldmann
- Division of Computer Assisted Medical Interventions (CAMI), German Cancer Research Center (DKFZ)
- Alejandro F. Frangi
- CISTIB - Center for Computational Imaging & Simulation Technologies in Biomedicine, The University of Leeds
- Peter M. Full
- Division of Computer Assisted Medical Interventions (CAMI), German Cancer Research Center (DKFZ)
- Bram van Ginneken
- Department of Radiology and Nuclear Medicine, Medical Image Analysis, Radboud University Center
- Allan Hanbury
- Institute of Information Systems Engineering, TU Wien
- Katrin Honauer
- Heidelberg Collaboratory for Image Processing (HCI), Heidelberg University
- Michal Kozubek
- Centre for Biomedical Image Analysis, Masaryk University
- Bennett A. Landman
- Electrical Engineering, Vanderbilt University
- Keno März
- Division of Computer Assisted Medical Interventions (CAMI), German Cancer Research Center (DKFZ)
- Oskar Maier
- Institute of Medical Informatics, Universität zu Lübeck
- Klaus Maier-Hein
- Division of Medical Image Computing (MIC), German Cancer Research Center (DKFZ)
- Bjoern H. Menze
- Institute for Advanced Studies, Department of Informatics, Technical University of Munich
- Henning Müller
- Information System Institute, HES-SO
- Peter F. Neher
- Division of Medical Image Computing (MIC), German Cancer Research Center (DKFZ)
- Wiro Niessen
- Departments of Radiology, Nuclear Medicine and Medical Informatics, Erasmus MC
- Nasir Rajpoot
- Department of Computer Science, University of Warwick
- Gregory C. Sharp
- Department of Radiation Oncology, Massachusetts General Hospital
- Korsuk Sirinukunwattana
- Institute of Biomedical Engineering, University of Oxford
- Stefanie Speidel
- Division of Translational Surgical Oncology (TCO), National Center for Tumor Diseases Dresden
- Christian Stock
- Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ)
- Danail Stoyanov
- Centre for Medical Image Computing (CMIC) & Department of Computer Science, University College London
- Abdel Aziz Taha
- Data Science Studio, Research Studios Austria FG
- Fons van der Sommen
- Department of Electrical Engineering, Eindhoven University of Technology
- Ching-Wei Wang
- AIExplore, NTUST Center of Computer Vision and Medical Imaging, Graduate Institute of Biomedical Engineering, National Taiwan University of Science and Technology
- Marc-André Weber
- Institute of Diagnostic and Interventional Radiology, University Medical Center Rostock
- Guoyan Zheng
- Institute for Surgical Technology and Biomechanics, University of Bern
- Pierre Jannin
- Univ Rennes, Inserm, LTSI (Laboratoire Traitement du Signal et de l’Image) - UMR_S 1099
- Annette Kopp-Schneider
- Division of Biostatistics, German Cancer Research Center (DKFZ)
- DOI
- https://doi.org/10.1038/s41467-018-07619-7
- Journal volume & issue
-
Vol. 9,
no. 1
pp. 1 – 13
Abstract
Biomedical image analysis challenges have increased in the last ten years, but common practices have not been established yet. Here the authors analyze 150 recent challenges and demonstrate that outcome varies based on the metrics used and that limited information reporting hampers reproducibility.
