Intrinsic judgment error in men’s championship world surf league: WSL 2021

Abstract

Introduction: Surfers’ performances are subjectively ranked by 5 judges. Low reliability and validity in judgment may lead to preventable errors and unfair scores. Objective: The aims of the present study is to describe the judgment error for each wave surfed in world surf league (WSL) championship tour of 2021, establish between-judge reliability, and establish judgment validity comparing each judge score with the final control score of the wave. Methodology: To describe the judgment error, we analyzed the available WSL data related to the 2021 Men’s Championship Tour (4,095 waves; 20,475 scores). Results: We found an inverted ‘U’-shaped pattern for the judgment error score vs. control score, explained by a quadratic regression model (R = 0.52; SEE = 0.10). The reliability produced an excellent intraclass correlation coefficient (CI95% = 0.97, 1.00), with a between judge (typical) error of 0.15. Validity analyses indicated a minimal real difference of 0.49 in the sum of two waves between the surfers for having 95% certainty for the heat winner. Conclusion: We recommend WSL incorporate intrinsic judgment errors to increase fairness and trust in the WSL championship tour.

Keywords

• Reliability
• Validity
• Surfing
• Judgment
• Fair game