Abstract Background To assess the inter-observer and intra-observer reliability of the magnetic resonance urography (MRU)—upper urinary tract dilation (UUTD) grading system. Methods A total of 40 patients with a diagnosis of NB were enrolled in this study. The images were assembled in an electronic presentation randomly. The presentations were reviewed and graded by 4 junior and 4 senior urologists. One week later, the images were randomized again and reassessed. The inter-observer reliability was estimated by Kendall’s coefficient of concordance and intra-class correlation coefficient (ICC), and the intra-observer reliability was estimated by weighted Cohen’s kappa. Results The inter-observer reliability strength was excellent for all urologists, with the ICC value of 0.939 (0.908–0.963) and Kendall’s W value of 0.967. The highest agreement was shown in Grade 4 at 92.50%, and the lowest in Grade 2 at 82.14%. All disagreements were within one grade of difference. Moreover, the Intra-observer reliability was excellent, with the weighted kappa value ranging from 0.904 to 0.954. Conclusions The inter-observer and intra-observer reliability of this novel MRU-UUTD grading system is confirmed, providing adequate evidence for broader clinical application.
