Comparison of inter- and intra-observer variability of breast density assessments using the fourth and fifth editions of Breast Imaging Reporting and Data System

Abstract

Background: Breast density is a well-known independent risk factor for breast cancer and can significantly affect the sensitivity of screening mammograms. Objective: We aimed to evaluate the intra- and inter-observer consistencies of breast density assessments using methods outlined in the fourth and fifth editions of the American College of Radiology (ACR) Breast Imaging Reporting and Data System (BI-RADS) guidelines to determine which method is more reliable. Materials and methods: Three radiologists with subspecialties in breast imaging defined breast density in 72 mammograms four times each: twice using the fourth edition of the ACR BI-RADS guidelines and twice using the fifth edition. The intra- and inter-observer agreements were calculated and compared for each method. Results: The weighted kappa values for the overall intra-observer agreement were 0.955 (95% confidence interval [CI]: 0.931–0.980) and 0.938 (95% CI: 0.907–0.968) when breast densities were assessed according to criteria outlined in the fourth and fifth ACR BI-RADS editions, respectively. The difference between these values was not statistically significant (p = .4). The overall Fleiss-Cohen (quadratic) weighted kappa for inter-observer agreement were 0.623 (95% CI: 0.517–0.729) and 0.702 (95% CI: 0.589–0.815) when breast densities were assessed according to criteria outlined in the fourth and fifth ACR BI-RADS editions, respectively. The difference between these values was not statistically significant (p = .32). Similarly, there were no significant differences in the evaluation of breast density (overall) when comparing breast density assignment using criteria outlined in the fourth and fifth ACR BI-RADS edition (p = .582). Conclusion: The ACR BI-RADS guideline is an acceptable method to classify breast density, resulting in substantial inter-observer agreements using criteria outlined in both the fourth and fifth editions. The intra-observer agreement was nearly perfect for radiologists using criteria outlined in both sets of guidelines. Moreover, although the percentage of women who were classified as having dense breasts was higher when radiologists used the fifth edition of ACR BI-RADS guidelines than when they used the fourth edition, this difference was not statistically significant. Keywords: Inter-observer variability, Intra-observer variability, Breast density, Mammography