Using Item Response Theory to evaluate the psychometric characteristics of questions in a Brazilian programme and the performance of dental teams in primary care.

Abstract

ObjectivesFirst, to assess the psychometric properties of key questions included in a public sector evaluation of primary dental care in Brazil; and second, to evaluate the performance of dental teams in relation to these items.MethodsSecondary analysis of a national primary care dataset monitoring quality and access to dental care. Data were collected through face-to-face interviews with representatives of dental teams participating in the 'National Programme for Improving Access and Quality of Primary Care'. Twenty-three mandatory questions about the dentists' reported delivery of dental procedures were included in the analysis. Item Response Theory (IRT) modelling was applied to measure the psychometric properties of the instrument-level of difficulty and discrimination parameter of each item-and then to estimate dental team performance scores based on these parameters. Based on IRT, possible scores ranged from -4 to +4.ResultsThree of the 23 mandatory items were removed due to poor internal consistency, resulting in a scale of 20 items for assessing dental team performance. The results showed variation in procedures delivered by the dental teams; whilst more than a half of the procedures were executed by at least 80% of the dental teams, those relating to dentures (partial/total) and frenectomy (lingual/labial) were performed by less than 30%. Amongst the 20 items included in the model, those related to partial/total dentures and oral cancer follow-up presented higher levels of difficulty and were less frequently provided. The items relating to the treatment of deciduous teeth and access to the dental pulp of permanent teeth had the highest discrimination parameters and, consequently, greater weight in the performance's score estimation; therefore, dental teams that did not perform these items had the lowest performance scores. In the present study, dental team performance scores ranged from -3.66 to +1.87 with a mean/median of -0.06/+0.01.ConclusionThe findings suggest that whilst the items within the instrument demonstrated some potential to discriminate between poor and very poor teams, they were ineffective in discriminating between poor and good teams. Whilst Brazilian dental teams perform many mandatory procedures, variation in the nature of their delivery of care requires further investigation to enhance service provision to the population.