Reliability testing of a modified MISTELS score using a low-cost trainer box

Abstract

Abstract Background Training programs such as the fundamentals of laparoscopic surgery (FLS) that are based on simulation are being currently used in several western countries. FLS allows skill acquisition and evaluation of competency in laparoscopic surgery. On the practical side, evaluation is determined by the MISTELS metrics (MISTELS is the acronym for the McGill inanimate system for training and evaluation of laparoscopic skills). This training program may be modified so that it can be implemented in countries with limited resources using a low-cost trainer box. Would the use of a low-cost trainer box alter the reliability of the MISTELS score? Objective of study The aim of the study was to evaluate the reliability of a modified MISTELS using a low-cost trainer box. Methods It was a prospective study carried out at Habib Thameur hospital in Tunis (Tunisia), between April 2016 and August 2016. The study involved residents from different surgical specialties in the departments of general surgery and paediatric surgery of the hospital during 2015 and 2016. This study assessed the reliability of a modified MISTELS system (Only three tasks were performed out of the five tasks used in the original MISTELS system). Evaluation was based on Cronbach’s alpha and intraclass correlation coefficients (ICC). A low-cost trainer box was designed and constructed. The residents included in the study performed three series of three tasks using this trainer box. The first series was scored by two trained raters to evaluate inter-rater reliability. The two-other series were successively performed to evaluate test-retest reliability. Results The internal consistency, assessed by Cronbach’s alpha, was at 0.929 which is an acceptable score. As for inter-rater and test-retest reliabilities that were assessed by ICCs, they yielded excellent scores that were at 1 and 0.95 (95% CI, 0.891–0.978) respectively. Conclusions The reliability of a modified MISTELS is not altered by the use of a low-cost trainer box. The score of the modified MISTELS is a reliable score for evaluating technical skills of surgical residents using a low-cost trainer box.

Keywords

• Training
• Evaluation
• Simulation
• FLS
• MISTELS
• Reliability