Inter-rater reliability of welfare outcome assessment by an expert and farmers of South Tyrolean dairy farming

Abstract

The implementation of an animal welfare assurance programme for dairy cattle in South Tyrol (Eastern Italian Alps) faces particular feasibility constraints due to the outstanding volume of travel associated with routine on-farm audits of remote mountain farms. Therefore, this study aims to estimate the inter-rater reliability of the expert’s and farmers’ welfare outcome assessment regarding recommendations to involve milk producers in animal welfare assurance within South Tyrolean dairy farming. A formal training programme containing a classroom session and an on-farm observation became mandatory for all 188 participating farmers, which was offered by the expert, applied as reference standard. On-farm data collected on the farmers’ cows (data set of 1719 dairy cows) were compared at animal level. Cohen’s kappa, respectively, weighted kappa, examined for several welfare indicators, range from slight to moderate agreement (κ = 0.018 − 0.416; κω = 0.163 − 0.310). These findings are further confirmed by results at farm level (ICC = 0.018 − 0.577). Continuous repeatability checks as part of routine audits are therefore proposed to substantially reduce the variability between the raters and to avoid significant bias in the welfare outcome assessment. In this way, the competence for regular and standardised monitoring could be increasingly transferred to dairy farmers in order to reduce the need for costly and time-consuming inspections by external auditors, which are in long-term perspective also harmful to the alpine environment. Additionally, the promotion of welfare assessment as an instructive management tool would intensify farmers’ commitment to the assessment process.Highlights Farmers’ self-assessment of welfare outcomes is cost-effective and eco-friendly, but reliability must be ensured. Inter-rater reliability of welfare outcome assessment by an expert and farmers presented a slight to moderate level. Repeatability assessment at regular intervals is proposed to reduce data variability and, thus, prevent bias in the welfare outcome assessment.

Keywords

• inter-rater reliability
• welfare outcome assessment
• self-assessment
• animal welfare assurance
• dairy cow