Statistical evaluation of corner detectors: does the statistical test have an effect?

Abstract

This study explores the use of several non‐parametric statistical tests for evaluating the performances of computer vision algorithms, specifically corner detectors, as a more reliable alternative to the graphical approaches that have been commonly employed to date. Using synthetic images carrying corners of different internal angles and orientations and a carefully designed testing framework, a ranking of the performances of corner detectors was established. It was found that Harris & Stephens and SUSAN out‐performed more modern detectors. These are one of the few examples where evaluation of vision operators independent of the application has predicted performance in a real‐world problem. A similar exercise on real images of the same patterns produced similar results and the findings of a real‐world application that uses corners to identify signage were also consistent. Together, all of the tests considered essentially perform pairwise comparisons of performance, so when many algorithms are involved it is important to take account of the potential for type I statistical errors. Several approaches were evaluated and none were found to affect the conclusions.

Keywords

• statistical evaluation
• corner detectors
• nonparametric statistical tests
• computer vision algorithms
• performance evaluation
• graphical approaches