Estimating measures of latent variables from m-alternative forced choice responses.

Abstract

Signal Detection Theory is the standard method used in psychophysics to estimate person ability in m-alternative forced choice tasks where stimuli are typically generated with known physical properties (e.g., size, frequency, contrast, etc …) and lie at known locations on a physical measurement axis. In contrast, variants of Item Response Theory are preferred in fields such as medical research and educational testing where the axis locations of items on questionnaires or multiple choice tests are not defined by any observable physical property and are instead defined by a latent (or unobservable) variable. We provide an extension of Signal Detection Theory to latent variables that employs the same strategy used in Item Response Theory and demonstrate the practical utility of our method by applying it to a set of clinically relevant face perception tasks with visually impaired individuals as subjects. A key advantage of our approach is that Signal Detection Theory explicitly models the m-alternative forced choice task while Item Response Theory does not. We show that Item Response Theory is inconsistent with key assumptions of the m-alternative forced choice task and is not a valid model for this paradigm. However, the simplest Item Response Theory model-the dichotomous Rasch model-is found to be a special case of SDT and provides a good approximation as long as the number of response alternatives m is small and remains fixed for all items.