Calibration and validation of an item bank for measuring general physical function of patients in medical rehabilitation settings

Abstract

Karon F Cook,1 Michael A Kallen,1 Deanna Hayes,2 Daniel Deutscher,3 Julie M Fritz,4,5 Mark W Werneke,6 Jerome E Mioduski2 1Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA; 2Focus On Therapeutic Outcomes, Inc., Knoxville, TN, USA; 3Department of Physical Therapy, Maccabi Healthcare Services, Tel-Aviv, Israel; 4Department of Orthopaedics, University of Utah School of Medicine, Salt Lake City, UT, USA; 5Department of Physical Therapy and Athletic Training, University of Utah School of Medicine, Salt Lake City, UT, USA; 6Private Practice, Waipahu, HI, USA Objective: The objective of this study was to report the item response theory (IRT) calibration of an 18-item bank to measure general physical function (GPF) in a wide range of conditions and evaluate the validity of the derived scores. Methods: All 18 items were administered to a large sample of patients (n=2337) who responded to the items in the context of their outpatient rehabilitation care. The responses, collected 1997–2000, were modeled using the graded response model, an IRT model appropriate for items with two or more response options. Inter-item consistency was evaluated based on Cronbach’s alpha and item to total correlations. Validity of scores was evaluated based on known-groups comparisons (age, number of health problems, symptom severity). The strength of a single, general factor was evaluated using a bi-factor model. Results were used to evaluate IRT assumption and as an indicator of construct validity. Local independence of item responses was also evaluated. Results: Response data met the assumptions of unidimensionality and local independence. Explained common variance of a single general factor was 0.88 (omega hierarchical =0.86). Only two of the 153 pairs of item residuals were flagged for local dependence. Inter-item consistency was high (0.93) as were item to total correlations (mean =0.61). Substantial variation was found in both IRT location (difficulty) and discrimination parameters. All omnibus known-groups comparisons were statistically significant (p<0.001).Conclusion: Item responses fit the IRT unidimensionality assumptions and were internally consistent. The usefulness of GPF scores in discriminating among patients with different levels of physical function was confirmed. Future studies should evaluate the validity of GPF scores based on an adaptive administration of items. Keywords: item response theory, computerized adaptive testing, rehabilitation, functional status, patient-reported outcomes

Keywords

• item response theory
• computerized adaptive testing
• rehabilitation
• functional status
• patient-reported outcomes