EBioMedicine (Jul 2015)
Crowdsourcing the General Public for Large Scale Molecular Pathology Studies in Cancer
- Francisco J. Candido dos Reis,
- Stuart Lynn,
- H. Raza Ali,
- Diana Eccles,
- Andrew Hanby,
- Elena Provenzano,
- Carlos Caldas,
- William J. Howat,
- Leigh-Anne McDuffus,
- Bin Liu,
- Frances Daley,
- Penny Coulson,
- Rupesh J. Vyas,
- Leslie M. Harris,
- Joanna M. Owens,
- Amy F.M. Carton,
- Janette P. McQuillan,
- Andy M. Paterson,
- Zohra Hirji,
- Sarah K. Christie,
- Amber R. Holmes,
- Marjanka K. Schmidt,
- Montserrat Garcia-Closas,
- Douglas F. Easton,
- Manjeet K. Bolla,
- Qin Wang,
- Javier Benitez,
- Roger L. Milne,
- Arto Mannermaa,
- Fergus Couch,
- Peter Devilee,
- Robert A.E.M. Tollenaar,
- Caroline Seynaeve,
- Angela Cox,
- Simon S. Cross,
- Fiona M. Blows,
- Joyce Sanders,
- Renate de Groot,
- Jonine Figueroa,
- Mark Sherman,
- Maartje Hooning,
- Hermann Brenner,
- Bernd Holleczek,
- Christa Stegmaier,
- Chris Lintott,
- Paul D.P. Pharoah
Affiliations
- Francisco J. Candido dos Reis
- Department of Oncology, University of Cambridge, Cambridge, UK
- Stuart Lynn
- Department of Physics (Astrophysics), University of Oxford, Oxford, UK
- H. Raza Ali
- Cancer Research UK, Cambridge Institute, Cambridge, UK
- Diana Eccles
- University of Southampton, Southampton, UK
- Andrew Hanby
- University of Leeds, Leeds, UK
- Elena Provenzano
- Addenbrookes Hospital NHS Trust, Cambridge, UK
- Carlos Caldas
- Cancer Research UK, Cambridge Institute, Cambridge, UK
- William J. Howat
- Cancer Research UK, Cambridge Institute, Cambridge, UK
- Leigh-Anne McDuffus
- Cancer Research UK, Cambridge Institute, Cambridge, UK
- Bin Liu
- Cancer Research UK, Cambridge Institute, Cambridge, UK
- Frances Daley
- Institute of Cancer Research, London, UK
- Penny Coulson
- Institute of Cancer Research, London, UK
- Rupesh J. Vyas
- Cancer Research UK, London, UK
- Leslie M. Harris
- Cancer Research UK, London, UK
- Joanna M. Owens
- Cancer Research UK, London, UK
- Amy F.M. Carton
- Cancer Research UK, London, UK
- Janette P. McQuillan
- Cancer Research UK, London, UK
- Andy M. Paterson
- Cancer Research UK, London, UK
- Zohra Hirji
- Cancer Research UK, London, UK
- Sarah K. Christie
- Cancer Research UK, London, UK
- Amber R. Holmes
- Cancer Research UK, London, UK
- Marjanka K. Schmidt
- Netherlands Cancer Institute, Antoni van Leeuwenhoek Hospital, Amsterdam, The Netherlands
- Montserrat Garcia-Closas
- Institute of Cancer Research, London, UK
- Douglas F. Easton
- Department of Oncology, University of Cambridge, Cambridge, UK
- Manjeet K. Bolla
- Department of Public Health and Primary Care, University of Cambridge, Cambridge, UK
- Qin Wang
- Department of Public Health and Primary Care, University of Cambridge, Cambridge, UK
- Javier Benitez
- Human Genotyping (CEGEN) Unit, Human Cancer Genetics Program, Spanish National Cancer Research Centre (CNIO), Madrid, Spain
- Roger L. Milne
- Cancer Epidemiology Centre, Cancer Council Victoria, Melbourne, Australia
- Arto Mannermaa
- Institute of Clinical Medicine, Pathology and Forensic Medicine, University of Eastern Finland, Finland
- Fergus Couch
- Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, USA
- Peter Devilee
- Department of Human Genetics & Department of Pathology, Leiden University Medical Center, Leiden, The Netherlands
- Robert A.E.M. Tollenaar
- Department of Surgical Oncology, Leiden University Medical Center, Leiden, The Netherlands
- Caroline Seynaeve
- Department of Medical Oncology, Family Cancer Clinic, Erasmus MC Cancer Institute, Rotterdam, The Netherlands
- Angela Cox
- Sheffield Cancer Research, Department of Oncology, University of Sheffield, Sheffield, UK
- Simon S. Cross
- Academic Unit of Pathology, Department of Neuroscience, University of Sheffield, Sheffield, UK
- Fiona M. Blows
- Department of Oncology, University of Cambridge, Cambridge, UK
- Joyce Sanders
- Netherlands Cancer Institute, Antoni van Leeuwenhoek Hospital, Amsterdam, The Netherlands
- Renate de Groot
- Netherlands Cancer Institute, Antoni van Leeuwenhoek Hospital, Amsterdam, The Netherlands
- Jonine Figueroa
- National Cancer Institute, USA
- Mark Sherman
- National Cancer Institute, USA
- Maartje Hooning
- Department of Medical Oncology, Family Cancer Clinic, Erasmus MC Cancer Institute, Rotterdam, The Netherlands
- Hermann Brenner
- Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Heidelberg, Germany
- Bernd Holleczek
- Saarland Cancer Registry, Saarbrücken, Germany
- Christa Stegmaier
- Saarland Cancer Registry, Saarbrücken, Germany
- Chris Lintott
- Department of Physics (Astrophysics), University of Oxford, Oxford, UK
- Paul D.P. Pharoah
- Department of Oncology, University of Cambridge, Cambridge, UK
- DOI
- https://doi.org/10.1016/j.ebiom.2015.05.009
- Journal volume & issue
-
Vol. 2,
no. 7
pp. 681 – 689
Abstract
Background: Citizen science, scientific research conducted by non-specialists, has the potential to facilitate biomedical research using available large-scale data, however validating the results is challenging. The Cell Slider is a citizen science project that intends to share images from tumors with the general public, enabling them to score tumor markers independently through an internet-based interface. Methods: From October 2012 to June 2014, 98,293 Citizen Scientists accessed the Cell Slider web page and scored 180,172 sub-images derived from images of 12,326 tissue microarray cores labeled for estrogen receptor (ER). We evaluated the accuracy of Citizen Scientist's ER classification, and the association between ER status and prognosis by comparing their test performance against trained pathologists. Findings: The area under ROC curve was 0.95 (95% CI 0.94 to 0.96) for cancer cell identification and 0.97 (95% CI 0.96 to 0.97) for ER status. ER positive tumors scored by Citizen Scientists were associated with survival in a similar way to that scored by trained pathologists. Survival probability at 15 years were 0.78 (95% CI 0.76 to 0.80) for ER-positive and 0.72 (95% CI 0.68 to 0.77) for ER-negative tumors based on Citizen Scientists classification. Based on pathologist classification, survival probability was 0.79 (95% CI 0.77 to 0.81) for ER-positive and 0.71 (95% CI 0.67 to 0.74) for ER-negative tumors. The hazard ratio for death was 0.26 (95% CI 0.18 to 0.37) at diagnosis and became greater than one after 6.5 years of follow-up for ER scored by Citizen Scientists, and 0.24 (95% CI 0.18 to 0.33) at diagnosis increasing thereafter to one after 6.7 (95% CI 4.1 to 10.9) years of follow-up for ER scored by pathologists. Interpretation: Crowdsourcing of the general public to classify cancer pathology data for research is viable, engages the public and provides accurate ER data. Crowdsourced classification of research data may offer a valid solution to problems of throughput requiring human input.
Keywords
