Healthcare Complaints Analysis Tool: reliability testing on a sample of Danish patient compensation claims

Abstract

Objective The study aim was to test the intra-assessor and interassessor reliability of the Healthcare Complaints Analysis Tool (HCAT) for categorising the information in the claim letters in a sample of Danish patient compensation claims.Design, setting and participants We used a random sample of 140 compensation cases completed by the Danish Patient Compensation Association that were filed in the field of acute medicine at Danish hospitals from 2007 to 2018. Four assessors were trained in using the HCAT manual before assessing the claim letters independently.Main outcome measures Intra-assessor and interassessor reliability was tested at domain, problem category and subcategory levels of the HCAT. We also investigated the reliability of ratings on the level of harm and of the descriptive details contained in the claim letters.Results The HCAT was reliable for identifying problem categories, with reliability scores ranging from 0.55 to 0.99. Reliability was lower when coding the ‘severity’ of the problem. Interassessor reliability was generally lower than intra-assessor reliability. The categories of ‘quality’ and ‘safety’ were the least reliable of the seven HCAT problem categories. Reliability at the subcategory level was generally satisfactory, with only a few subcategories having poor reliability. Reliability was at least moderate when coding the stage of care, the complainant and the staff group involved. However, the coding of ‘level of harm’ was found to be unreliable (intrareliability 0.06; inter-reliability 0.29).Conclusion Overall, HCAT was found to be a reliable tool for categorising problem types in patient compensation claims.