Novel Statistical Regularized Extreme Learning Algorithm to Address the Multicollinearity in Machine Learning

Abstract

The multicollinearity problem is a common phenomenon in data-driven studies, significantly affecting the performance of machine learning algorithms during the process of extracting information from data. Despite its widespread use across various fields, the extreme learning machine (ELM) also suffers from multicollinearity issues. To address this challenge, the ridge and Liu estimators, drawn from statistics literature, have been integrated into ELM theory, resulting in a notable advancement. This study aims to further enhance the capabilities of ridge and Liu estimators within the ELM framework by introducing two innovative two-parameter algorithms (TP1-ELM and TP2-ELM) that simultaneously incorporate both estimators. The proposed algorithms undergo comprehensive benchmarking against ELM, ELM-based algorithms, and other commonly used machine learning techniques across seven diverse datasets. Benchmark results demonstrate that the proposed algorithms consistently outperform both ELM-focused approaches and traditional machine learning algorithms on most datasets, yielding more generalizable and stable results. These findings suggest that the proposed algorithms offer a promising alternative to traditional machine learning techniques for regression and classification tasks, particularly in scenarios where multicollinearity is a concern.

Keywords

• Extreme learning machine
• Liu estimator
• machine learning
• multicollinearity
• ridge estimator
• Tikhonov regularization