Frontiers in Psychology (May 2018)
Manipulating the Alpha Level Cannot Cure Significance Testing
- David Trafimow,
- Valentin Amrhein,
- Valentin Amrhein,
- Corson N. Areshenkoff,
- Carlos J. Barrera-Causil,
- Eric J. Beh,
- Yusuf K. Bilgiç,
- Roser Bono,
- Roser Bono,
- Michael T. Bradley,
- William M. Briggs,
- Héctor A. Cepeda-Freyre,
- Sergio E. Chaigneau,
- Daniel R. Ciocca,
- Juan C. Correa,
- Denis Cousineau,
- Michiel R. de Boer,
- Subhra S. Dhar,
- Igor Dolgov,
- Juana Gómez-Benito,
- Juana Gómez-Benito,
- Marian Grendar,
- Marian Grendar,
- James W. Grice,
- Martin E. Guerrero-Gimenez,
- Andrés Gutiérrez,
- Tania B. Huedo-Medina,
- Klaus Jaffe,
- Armina Janyan,
- Armina Janyan,
- Ali Karimnezhad,
- Fränzi Korner-Nievergelt,
- Fränzi Korner-Nievergelt,
- Koji Kosugi,
- Martin Lachmair,
- Rubén D. Ledesma,
- Rubén D. Ledesma,
- Roberto Limongi,
- Roberto Limongi,
- Marco T. Liuzza,
- Rosaria Lombardo,
- Michael J. Marks,
- Gunther Meinlschmidt,
- Gunther Meinlschmidt,
- Gunther Meinlschmidt,
- Ladislas Nalborczyk,
- Ladislas Nalborczyk,
- Hung T. Nguyen,
- Raydonal Ospina,
- Jose D. Perezgonzalez,
- Roland Pfister,
- Juan J. Rahona,
- David A. Rodríguez-Medina,
- Xavier Romão,
- Susana Ruiz-Fernández,
- Susana Ruiz-Fernández,
- Susana Ruiz-Fernández,
- Isabel Suarez,
- Marion Tegethoff,
- Mauricio Tejo,
- Rens van de Schoot,
- Rens van de Schoot,
- Ivan I. Vankov,
- Santiago Velasco-Forero,
- Tonghui Wang,
- Yuki Yamada,
- Felipe C. M. Zoppino,
- Fernando Marmolejo-Ramos
Affiliations
- David Trafimow
- Department of Psychology, New Mexico State University, Las Cruces, NM, United States
- Valentin Amrhein
- Zoological Institute, University of Basel, Basel, Switzerland
- Valentin Amrhein
- Swiss Ornithological Institute, Sempach, Switzerland
- Corson N. Areshenkoff
- Centre for Neuroscience Studies, Queens University, Kingston, ON, Canada
- Carlos J. Barrera-Causil
- Faculty of Applied and Exact Sciences, Metropolitan Technological Institute, Medellín, Colombia
- Eric J. Beh
- School of Mathematical and Physical Sciences, University of Newcastle, Callaghan, NSW, Australia
- Yusuf K. Bilgiç
- Department of Mathematics, State University of New York at Geneseo, Geneseo, NY, United States
- Roser Bono
- Quantitative Psychology Unit, Faculty of Psychology, University of Barcelona, Barcelona, Spain
- Roser Bono
- Institut de Neurociències, University of Barcelona, Barcelona, Spain
- Michael T. Bradley
- 0Department of Psychology, Faculty of Arts, University of New Brunswick, Saint John, NB, Canada
- William M. Briggs
- 1Independent Researcher, New York, NY, United States
- Héctor A. Cepeda-Freyre
- 2School of Psychology, Benemérita Universidad Autónoma de Puebla, Puebla, Mexico
- Sergio E. Chaigneau
- 3Center for Social and Cognitive Neuroscience, School of Psychology, Universidad Adolfo Ibáñez, Santiago, Chile
- Daniel R. Ciocca
- 4Oncology Laboratory, Instituto de Medicina y Biologia Experimental de Cuyo, CCT CONICET Mendoza, Mendoza, Argentina
- Juan C. Correa
- 5School of Statistics, Faculty of Sciences, National University of Colombia, Medellín, Colombia
- Denis Cousineau
- 6School of Psychology, University of Ottawa, Ottawa, ON, Canada
- Michiel R. de Boer
- 7Department of Health Sciences, Vrije Universiteit Amsterdam and Amsterdam Public Health Research Institute, Amsterdam, Netherlands
- Subhra S. Dhar
- 8Department of Mathematics and Statistics, Indian Institute of Technology, Kanpur, India
- Igor Dolgov
- Department of Psychology, New Mexico State University, Las Cruces, NM, United States
- Juana Gómez-Benito
- Quantitative Psychology Unit, Faculty of Psychology, University of Barcelona, Barcelona, Spain
- Juana Gómez-Benito
- Institut de Neurociències, University of Barcelona, Barcelona, Spain
- Marian Grendar
- 9Biomedical Center Martin, Jessenius Faculty of Medicine, Comenius University, Martin, Slovakia
- Marian Grendar
- 0Institute of Measurement Science, Slovak Academy of Sciences, Bratislava, Slovakia
- James W. Grice
- 1Department of Psychology, Oklahoma State University, Stillwater, OK, United States
- Martin E. Guerrero-Gimenez
- 4Oncology Laboratory, Instituto de Medicina y Biologia Experimental de Cuyo, CCT CONICET Mendoza, Mendoza, Argentina
- Andrés Gutiérrez
- 2Faculty of Statistics, Saint Thomas University, Bogotá, Colombia
- Tania B. Huedo-Medina
- 3Department of Allied Health Sciences, College of Health, Agriculture, and Natural Resources, University of Connecticut, Storrs, CT, United States
- Klaus Jaffe
- 4Departamento de Biología de Organismos, Universidad Simón Bolívar, Caracas, Venezuela
- Armina Janyan
- 5Department of Cognitive Science and Psychology, New Bulgarian University, Sofia, Bulgaria
- Armina Janyan
- 6National Research Tomsk State University, Tomsk, Russia
- Ali Karimnezhad
- 7Department of Biochemistry, Microbiology, and Immunology, University of Ottawa, Ottawa, ON, Canada
- Fränzi Korner-Nievergelt
- Swiss Ornithological Institute, Sempach, Switzerland
- Fränzi Korner-Nievergelt
- 8Oikostat GmbH, Ettiswil, Switzerland
- Koji Kosugi
- 9School of Human Sciences, Senshu University, Kawasaki, Japan
- Martin Lachmair
- 0Multimodal Interaction Lab, Leibniz-Institut für Wissensmedien, Tübingen, Germany
- Rubén D. Ledesma
- 1Consejo Nacional de Investigaciones Científicas y Técnicas, Mar del Plata, Argentina
- Rubén D. Ledesma
- 2Facultad de Psicología, Universidad Nacional de Mar del Plata, Mar del Plata, Argentina
- Roberto Limongi
- 3Pontificia Universidad Católica de Valparaíso, Valparaíso, Chile
- Roberto Limongi
- 4Vicerrectoría de Investigación y Desarrollo, Universidad Tecnológica de Chile INACAP, Santiago, Chile
- Marco T. Liuzza
- 5Department of Medical and Surgical Sciences, “Magna Graecia” University of Catanzaro, Catanzaro, Italy
- Rosaria Lombardo
- 6Economics Department, University of Campania “Luigi Vanvitelli”, Capua, Italy
- Michael J. Marks
- Department of Psychology, New Mexico State University, Las Cruces, NM, United States
- Gunther Meinlschmidt
- 7Department of Psychosomatic Medicine, University Hospital Basel and University of Basel, Basel, Switzerland
- Gunther Meinlschmidt
- 8Division of Clinical Psychology and Cognitive Behavioral Therapy, International Psychoanalytic University, Berlin, Germany
- Gunther Meinlschmidt
- 9Division of Clinical Psychology and Epidemiology, Department of Psychology, University of Basel, Basel, Switzerland
- Ladislas Nalborczyk
- 0Université Grenoble Alpes, Centre National de la Recherche Scientifique, LPNC, Grenoble, France
- Ladislas Nalborczyk
- 1Department of Experimental Clinical and Health Psychology, Ghent University, Ghent, Belgium
- Hung T. Nguyen
- 2Department of Mathematical Sciences, New Mexico State University, Las Cruces, NM, United States
- Raydonal Ospina
- 3Computational Statistics Laboratory (CAST), Department of Statistics, Universidade Federal de Pernambuco, Recife, Brazil
- Jose D. Perezgonzalez
- 4Business School, Massey University, Albany, New Zealand
- Roland Pfister
- 5Department of Psychology III, University of Würzburg, Würzburg, Germany
- Juan J. Rahona
- 0Multimodal Interaction Lab, Leibniz-Institut für Wissensmedien, Tübingen, Germany
- David A. Rodríguez-Medina
- 6School of Psychology, National Autonomous University of Mexico, Mexico City, Mexico
- Xavier Romão
- 7CONSTRUCT-LESE, Faculty of Engineering, University of Porto, Porto, Portugal
- Susana Ruiz-Fernández
- 0Multimodal Interaction Lab, Leibniz-Institut für Wissensmedien, Tübingen, Germany
- Susana Ruiz-Fernández
- 8FOM Hochschule für Oekonomie und Management, Essen, Germany
- Susana Ruiz-Fernández
- 9LEAD Graduate School & Research Network, University of Tübingen, Tübingen, Germany
- Isabel Suarez
- 0Department of Psychology, Universidad del Norte, Barranquilla, Colombia
- Marion Tegethoff
- 1Division of Clinical Psychology and Psychiatry, Department of Psychology, University of Basel, Basel, Switzerland
- Mauricio Tejo
- 2Facultad de Ciencias Naturales y Exactas, Universidad de Playa Ancha, Valparaíso, Chile
- Rens van de Schoot
- 3Department of Methods and Statistics, Faculty of Social and Behavioural Sciences, Utrecht University, Utrecht, Netherlands
- Rens van de Schoot
- 4North-West University, Optentia Research Focus Area, Vanderbijlpark, South Africa
- Ivan I. Vankov
- 5Department of Cognitive Science and Psychology, New Bulgarian University, Sofia, Bulgaria
- Santiago Velasco-Forero
- 5MINES Paristech, PSL Research University, Centre for Mathematical Morphology, Paris, France
- Tonghui Wang
- 6Department of Mathematical Sciences, New Mexico State University, Las Cruces, NM, United States
- Yuki Yamada
- 7Faculty of Arts and Science, Kyushu University, Fukuoka, Japan
- Felipe C. M. Zoppino
- 4Oncology Laboratory, Instituto de Medicina y Biologia Experimental de Cuyo, CCT CONICET Mendoza, Mendoza, Argentina
- Fernando Marmolejo-Ramos
- 8School of Psychology, The University of Adelaide, Adelaide, SA, Australia
- DOI
- https://doi.org/10.3389/fpsyg.2018.00699
- Journal volume & issue
-
Vol. 9
Abstract
We argue that making accept/reject decisions on scientific hypotheses, including a recent call for changing the canonical alpha level from p = 0.05 to p = 0.005, is deleterious for the finding of new discoveries and the progress of science. Given that blanket and variable alpha levels both are problematic, it is sensible to dispense with significance testing altogether. There are alternatives that address study design and sample size much more directly than significance testing does; but none of the statistical tools should be taken as the new magic method giving clear-cut mechanical answers. Inference should not be based on single studies at all, but on cumulative evidence from multiple independent studies. When evaluating the strength of the evidence, we should consider, for example, auxiliary assumptions, the strength of the experimental design, and implications for applications. To boil all this down to a binary decision based on a p-value threshold of 0.05, 0.01, 0.005, or anything else, is not acceptable.
Keywords
