Frontiers in Psychology (Apr 2018)
Empirical Tryout of a New Statistic for Detecting Temporally Inconsistent Responders
Abstract
Statistical screening of self-report data is often advised to support the quality of analyzed responses – For example, reduction of insufficient effort responding (IER). One recently introduced index based on Mahalanobis’s D for detecting outliers in cross-sectional designs replaces centered scores with difference scores between repeated-measure items: Termed person temporal consistency (D2ptc). Although the adapted D2ptc index demonstrated usefulness in simulation datasets, it has not been applied to empirical data. The current study addresses D2ptc’s low uptake by critically appraising its performance across three empirical applications. Independent samples were selected to represent a range of scenarios commonly encountered by organizational researchers. First, in Sample 1, a repeat-measure of future time perspective (FTP) inexperienced working adults (age >40-years; n = 620) indicated that temporal inconsistency was significantly related to respondent age and item reverse-scoring. Second, in repeat-measure of team efficacy aggregations, D2ptc successfully detected team-level inconsistency across repeat-performance cycles. Thirdly, the usefulness of the D2ptc was examined in an experimental study dataset of subjective life expectancy indicated significantly more stable responding in experimental conditions compared to controls. The empirical findings support D2ptc’s flexible and useful application to distinct study designs. Discussion centers on current limitations and further extensions that may be of value to psychologists screening self-report data for strengthening response quality and meaningfulness of inferences from repeated-measures self-reports. Taken together, the findings support the usefulness of the newly devised statistic for detecting IER and other extreme response patterns.
Keywords
