Brazilian Political Science Review (Apr 2007)

When is statistical significance not significant?

  • Dalson Britto Figueiredo Filho,
  • Ranulfo Paranhos,
  • Enivaldo C. da Rocha,
  • Mariana Batista,
  • José Alexandre da Silva Jr.,
  • Manoel L. Wanderley D. Santos,
  • Jacira Guiro Marino

Journal volume & issue
Vol. 7, no. 1
pp. 31 – 55

Abstract

Read online

The article provides a non-technical introduction to the p value statistics. Its main purpose is to help researchers make sense of the appropriate role of the p value statistics in empirical political science research. On methodological grounds, we use replication, simulations and observational data to show when statistical significance is not significant. We argue that: (1) scholars must always graphically analyze their data before interpreting the p value; (2) it is pointless to estimate the p value for non-random samples; (3) the p value is highly affected by the sample size, and (4) it is pointless to estimate the p value when dealing with data on population.

Keywords