Reliability of competency-based, multi-dimensional, multiple-rater performance ratings

Abstract

The purpose of this study was to test the assumption that the utilisation of competency-based, multi-dimensional, multiple-rater performance ratings would result in reliable and useful measurements of the performance of managers (N=200) in a specific South African organisation. Reliability coefficients were computed and factor analysis were undertaken to determine the reliability of the ratings. The results indicated high inter-method reliability and low inter-rater reliability. Although the competency- based, multi-dimensional, multiple-rater approach appeared to have facilitated valuable input towards the assessment process a greater degree of reliability, validity and usefulness was not necessarily achieved. Implications of these findings are discussed. Opsomming Die doel van die studie was om die aanname dat die gebruik van bevoegdheidsgebaseerde, multi-dimensionele, meervoudige-beoordelaar prestasiebeoordeling tot betroubare en bruikbare metings van die prestasie van bestuurders (N=200) in ’n spesifieke Suid-Afrikaanse organisasie sou lei, te toets. Betroubaarheidskoëffisiënte is bereken en faktorontledings is uitgevoer om die betroubaarhede te bepaal. Die resultate het hoë inter-metodiese en lae inter-beoordelaar betroubaarhede aangedui. Hoewel dit wou voorkom of die bevoegdheids gebaseerde, multidimensionele, meervoudige beoordelaar benadering tot prestasie beoordeling nuttige inligting aan die prestasie beoordelingsproses verskaf, is ’n verhoogde mate van betroubaarheid, geldigheid en ruikbaarheid nie noodwendig verkry nie. Implikasies van die bevindinge word bespreek.

Keywords

• Competency-based performance ratings
• Multi-dimensional performance ratings
• Multiple-rater performance ratings
• Performance ratings