Personnel Assessment and Decisions (Jul 2019)

Criterion-related Validity of Forced-Choice Personality Measures: A Cautionary Note Regarding Thurstonian IRT versus Classical Test Theory Scoring

  • Fisher, Peter,
  • Robie, Chet,
  • Christiansen, Neil,
  • Speer, Andrew,
  • Schneider, Leann

DOI
https://doi.org/10.25035/pad.2019.01.003
Journal volume & issue
Vol. 5, no. 1

Abstract

Read online

This study examined criterion-related validity for job-related composites of forced-choice personality scores against job performance using both Thurstonian Item Response Theory (TIRT) and Classical Test Theory (CTT) scoring methods. Correlations were computed across 11 different samples that differed in job or role within a job. A meta-analysis of the correlations (k = 11 and N = 613) found a higher average corrected correlation for CTT (mean ρ = .38) than for TIRT (mean ρ = .00). Implications and directions for future research are discussed.