Frontiers in Psychology (Sep 2015)

Meta-Analyses Are No Substitute for Registered Replications: A Skeptical Perspective on Religious Priming

  • Michiel eVan Elk,
  • Dora eMatzke,
  • Quentin eGronau,
  • Maime eGuang,
  • Joachim eVandekerckhove,
  • Eric-Jan eWagenmakers

DOI
https://doi.org/10.3389/fpsyg.2015.01365
Journal volume & issue
Vol. 6

Abstract

Read online

According to a recent meta-analysis, religious priming has a positive effect on prosocial behavior (Shariff et al., 2015). We first argue that this meta-analysis suffers from a number of methodological shortcomings that limit the conclusions that can be drawn about the potential benefits of religious priming. Next we present a re-analysis of the religious priming data using two different meta-analytic techniques. A PET-PEESE meta-analysis suggests that the effect of religious priming is driven solely by publication bias. In contrast, an analysis using Bayesian bias correction suggests the presence of a religious priming effect, even after controlling for publication bias. These contradictory statistical results demonstrate that meta-analytic techniques alone may not be sufficiently robust to firmly establish the presence or absence of an effect. We argue that a conclusive resolution of the debate about the effect of religious priming on prosocial behavior – and about theoretically disputed effects more generally – requires a large-scale, preregistered replication project, which we consider to be the sole remedy for the adverse effects of experimenter bias and publication bias.

Keywords