Patterns of Reliability: Assessing the Reproducibility and Integrity of DNA Methylation Measurement
Karen Sugden,
Eilis J. Hannon,
Louise Arseneault,
Daniel W. Belsky,
David L. Corcoran,
Helen L. Fisher,
Renate M. Houts,
Radhika Kandaswamy,
Terrie E. Moffitt,
Richie Poulton,
Joseph A. Prinz,
Line J.H. Rasmussen,
Benjamin S. Williams,
Chloe C.Y. Wong,
Jonathan Mill,
Avshalom Caspi
Affiliations
Karen Sugden
Department of Psychology and Neuroscience, Duke University, Grey Building, 2020 West Main Street, Suite 201, Durham, NC 27705, USA; Center for Genomic and Computational Biology, Duke University, Durham, NC, USA; Corresponding author
Eilis J. Hannon
Complex Disease Epigenetics Group, University of Exeter Medical School, Exeter, UK
Louise Arseneault
King's College London, Social, Genetic, and Developmental Psychiatry Research Centre, Institute of Psychiatry, Psychology, and Neuroscience, London, UK
Daniel W. Belsky
Department of Epidemiology & Butler Aging Center, Columbia University Mailman School of Public Health, New York, NY, USA
David L. Corcoran
Center for Genomic and Computational Biology, Duke University, Durham, NC, USA
Helen L. Fisher
King's College London, Social, Genetic, and Developmental Psychiatry Research Centre, Institute of Psychiatry, Psychology, and Neuroscience, London, UK
Renate M. Houts
Department of Psychology and Neuroscience, Duke University, Grey Building, 2020 West Main Street, Suite 201, Durham, NC 27705, USA
Radhika Kandaswamy
King's College London, Social, Genetic, and Developmental Psychiatry Research Centre, Institute of Psychiatry, Psychology, and Neuroscience, London, UK
Terrie E. Moffitt
Department of Psychology and Neuroscience, Duke University, Grey Building, 2020 West Main Street, Suite 201, Durham, NC 27705, USA; Center for Genomic and Computational Biology, Duke University, Durham, NC, USA; King's College London, Social, Genetic, and Developmental Psychiatry Research Centre, Institute of Psychiatry, Psychology, and Neuroscience, London, UK; Department of Psychiatry and Behavioral Sciences, Duke University School of Medicine, Durham, NC, USA
Richie Poulton
Dunedin Multidisciplinary Health and Development Research Unit, University of Otago, Dunedin, New Zealand
Joseph A. Prinz
Center for Genomic and Computational Biology, Duke University, Durham, NC, USA
Line J.H. Rasmussen
Department of Psychology and Neuroscience, Duke University, Grey Building, 2020 West Main Street, Suite 201, Durham, NC 27705, USA; Clinical Research Centre, Copenhagen University Hospital Amager and Hvidovre, Hvidovre, Denmark
Benjamin S. Williams
Department of Psychology and Neuroscience, Duke University, Grey Building, 2020 West Main Street, Suite 201, Durham, NC 27705, USA; Center for Genomic and Computational Biology, Duke University, Durham, NC, USA
Chloe C.Y. Wong
King's College London, Social, Genetic, and Developmental Psychiatry Research Centre, Institute of Psychiatry, Psychology, and Neuroscience, London, UK
Jonathan Mill
Complex Disease Epigenetics Group, University of Exeter Medical School, Exeter, UK
Avshalom Caspi
Department of Psychology and Neuroscience, Duke University, Grey Building, 2020 West Main Street, Suite 201, Durham, NC 27705, USA; Center for Genomic and Computational Biology, Duke University, Durham, NC, USA; King's College London, Social, Genetic, and Developmental Psychiatry Research Centre, Institute of Psychiatry, Psychology, and Neuroscience, London, UK; Department of Psychiatry and Behavioral Sciences, Duke University School of Medicine, Durham, NC, USA
Summary: DNA methylation plays an important role in both normal human development and risk of disease. The most utilized method of assessing DNA methylation uses BeadChips, generating an epigenome-wide “snapshot” of >450,000 observations (probe measurements) per assay. However, the reliability of each of these measurements is not equal, and little consideration is paid to consequences for research. We correlated repeat measurements of the same DNA samples using the Illumina HumanMethylation450K and the Infinium MethylationEPIC BeadChips in 350 blood DNA samples. Probes that were reliably measured were more heritable and showed consistent associations with environmental exposures, gene expression, and greater cross-tissue concordance. Unreliable probes were less replicable and generated an unknown volume of false negatives. This serves as a lesson for working with DNA methylation data, but the lessons are equally applicable to working with other data: as we advance toward generating increasingly greater volumes of data, failure to document reliability risks harming reproducibility. The Bigger Picture: Although DNA methylation data are used widely by researchers in many fields, the reliability of these data are surprisingly variable. Our findings remind us that, in an age of increasingly big data, research is only as robust as its foundations. We hope that our findings will improve the integrity of DNA methylation studies. We also hope that our findings serve as a cautionary reminder for those generating and implementing big data of any type: reliability is a fundamental aspect of replicability. Conducting analysis with reliable data will improve chances of replicable findings, which might lead to more actionable targets for further research. To the extent that reliable data improve replicability, the knock-on effect will be more public confidence in research and less effort spent trying to replicate findings that are bound to fail.
