Statistics for Evaluating Pre-post Change: Relation Between Change in the Distribution Center and Change in the Individual Scores

Abstract

Read online

In a number of scientific fields, researchers need to assess whether a variable has changed between two time points. Average-based change statistics (ABC) such as Cohen's d or Hays' ω2 evaluate the change in the distributions' center, whereas Individual-based change statistics (IBC) such as the Standardized Individual Difference or the Reliable Change Index evaluate whether each case in the sample experienced a reliable change. Through an extensive simulation study we show that, contrary to what previous studies have speculated, ABC and IBC statistics are closely related. The relation can be assumed to be linear, and was found regardless of sample size, pre-post correlation, and shape of the scores' distribution, both in single group designs and in experimental designs with a control group. We encourage other researchers to use IBC statistics to evaluate their effect sizes because: (a) they allow the identification of cases that changed reliably; (b) they facilitate the interpretation and communication of results; and (c) they provide a straightforward evaluation of the magnitude of empirical effects while avoiding the problems of arbitrary general cutoffs.

Keywords

• individual reliable change
• effect size estimation
• assessment of change
• Reliable Change Index (RCI)
• pre-post change