Insufficient Effort Responding in Surveys Assessing Self-Regulated Learning: Nuisance or Fatal Flaw?

Abstract

Despite concerns about their validity, self-report surveys remain the primary data collection method in the research of self-regulated learning (SRL). To address some of these concerns, we took a data set comprised of college students’ self-reported beliefs and behaviours related to SRL, assessed across three surveys, and examined it for instance of a specific threat to validity, insufficient effort responding (IER; Huang, Curran, Keeny, Poposki, & DeShon, 2012). Using four validated indicators of IER, we found the rate of IER to vary between 12-16%. Critically, while we found that students characterised as inattentive and attentive differed in some basic descriptive statistics, the inclusion of inattentive students within the data set did not alter more substantial inferences or conclusions drawn from the data. This study provides the first direct examination of the impact of respondents’ attention on the validity of SRL data generated from self-report surveys.