Mathematics (Nov 2021)

Redundancy Is Not Necessarily Detrimental in Classification Problems

  • Sebastián Alberto Grillo,
  • José Luis Vázquez Noguera,
  • Julio César Mello Román,
  • Miguel García-Torres,
  • Jacques Facon,
  • Diego P. Pinto-Roa,
  • Luis Salgueiro Romero,
  • Francisco Gómez-Vela,
  • Laura Raquel Bareiro Paniagua,
  • Deysi Natalia Leguizamon Correa

DOI
https://doi.org/10.3390/math9222899
Journal volume & issue
Vol. 9, no. 22
p. 2899

Abstract

Read online

In feature selection, redundancy is one of the major concerns since the removal of redundancy in data is connected with dimensionality reduction. Despite the evidence of such a connection, few works present theoretical studies regarding redundancy. In this work, we analyze the effect of redundant features on the performance of classification models. We can summarize the contribution of this work as follows: (i) develop a theoretical framework to analyze feature construction and selection, (ii) show that certain properly defined features are redundant but make the data linearly separable, and (iii) propose a formal criterion to validate feature construction methods. The results of experiments suggest that a large number of redundant features can reduce the classification error. The results imply that it is not enough to analyze features solely using criteria that measure the amount of information provided by such features.

Keywords