Journal of Advances in Medical Education and Professionalism (Jan 2016)
Investigation of outliers of evaluation scores among school of health instructors using outlier - determination indices
Abstract
Introduction: Teacher evaluation, as an important strategy for improving the quality of education, has been considered by universities and leads to a better understanding of the strengths and weaknesses of education. Analysis of instructors’ scores is one of the main fields of educational research. Since outliers affect analysis and interpretation of information processes both structurally and conceptually, understanding the methods of detecting outliers in collected data can be helpful for scholars, data analysts, and researchers. The present study aimed to present and compare the available techniques for detecting outliers. Methods: In this cross-sectional study, the statistical population included the evaluation forms of instructors completed by the students of Shiraz School of Health in the first and second semesters of the academic year 2012-2013. All the forms related to these years (N=1317) were entered into analysis through census. Then, four methods (Dixon, Gauss, Grubb, and Graphical methods) were used for determining outliers. Kappa coefficient was also used to determine the agreement among the methods. Results: In this study 1317 forms were completed by 203 undergraduate and 1114 postgraduate students. The mean scores given by undergraduates and postgraduates were 17.24±3.04 and 18.91±1.82, respectively. The results showed that Dixon and Grubb were the most appropriate methods to determine the outliers of evaluation scores in small samples, because they had appropriate agreement. On the other hand, NPP and QQ plot were the most appropriate methods in large samples. Conclusion: The results showed that each of the studied methods could help us, in some way, determine outliers. Researchers and analysts who intend to select and use the methods must first review the observations with the help of descriptive information and overview of the distribution. Determination of outliers is important in evaluation of instructors, because by determining the outliers and removing the data that might have been recorded incorrectly, more accurate and reliable results can be obtained.
