Developing Device-Equivalent and Effective Measures of Complex Thinking with an Information Processing Framework and Mobile First Design Principles

Abstract

Organizations are increasingly offering pre-employment assessments on mobile devices to evaluate candidates. The aim of this study is to investigate whether employing a mobile first responsive web design based on an information processing framework will result in device-equivalent measures of cognitive ability. Tests of numerical and deductive reasoning composed of interactive item types were tested for measurement equivalence across device types. Hypotheses were tested using data collected from paid participants over 3 weeks in 2018. Participants completed the test on both a PC and a mobile device. Paired samples t-tests indicated no mean differences in scores or number of items completed across device type. Additional analyses indicated that these item types demonstrated configural invariance and at least partial metric and scalar invariance across device types. The results of this study provide strong support that mobile first design can yield a valid and reliable test that can be used on any device and for any job level.