When is it most appropriate to control for initial scores? A comparison of examination methods for two-wave panel survey data changes

Abstract

For decades, researchers have proposed several methods to determine whether observed differences between two or more groups are actual changes or merely regression artifacts. Among other methods, the following three have been used most frequently to analyze the amount of change from pretest to posttest across groups in the field of psychology: ANOVA on the change score, ANCOVA, and analysis of residual change score. Through analysis, this study determines which method would be a better choice in the context of non-experimental survey research. Specifically, this study examines whether the scores of White Americans change more than those of Black Americans in the vocabulary test. Data for this study were taken from the General Social Survey (GSS) panel. The GSS contains a variable called Wordsum, which is a ten-word brief vocabulary test. The findings suggested that, among the three methods examined, the change score analysis would be the most desirable method when there are differences in initial scores between preexisting stable groups. More preferably, however, latent change score models should be used when possible. When there are no differences in initial scores between preexisting stable groups, the change score analysis, ANCOVA, and the residual change analysis would yield similar results.

Keywords

• regression to the mean
• change score analysis
• anova
• ancova
• latent change score models