Effectiveness of immediate vs. delayed recall in detecting invalid performance in coached and uncoached simulators: Results of two experimental studies

Abstract

Objective: Two experimental studies were conducted to compare the ability of immediate and delayed recall indicators to discriminate between performances of simulators and full-effort clinical and nonclinical participants. Methods: Three groups of simulators (uncoached, symptom-coached, and testcoached), one group of community controls, and one group of cognitively impaired patients were assessed with four experimental memory tests, in which the immediate and delayed recall tasks were separated by three other tasks. Results: Across both studies, delayed recall demonstrated higher accuracy than immediate recall in classifying simulated performances as invalid, as compared to performances of bona fide clinical participants. ROC curve results showed sensitivities below 50% for both indicators at specificities of ≥ 90%. Computing performance curves across recall trials revealed descending trends for all three simulator groups indicating a suppressed learning effect as a marker of noncredible performances. Among types of coaching, test-coaching proved to decrease differences between simulators and patients. Discussion: The effectiveness of such indicators in clinical evaluations and their vulnerability to information about test-taking strategies are discussed.

Keywords

• invalid performance
• validity indicators
• recall
• simulation design
• coaching