An Adaptive Learning Algorithm for Regularized Extreme Learning Machine

Abstract

Extreme learning machine (ELM) has become popular in recent years, due to its robust approximation capacity and fast learning speed. It is common to add a $\ell _{2}$ penalty term in basic ELM to avoid over-fitting. However, in $\ell _{2}$ -regularized extreme learning machine ( $\ell _{2}$ -RELM), choosing a suitable regularization factor is random and time consuming. In order to select a satisfactory regularization factor automatically, we proposed an adaptive regularized extreme learning machine (A-RELM) by replacing the regularization factor with a function. The function is defined in terms of the output weights named regularization function. And an iterative algorithm is proposed for obtaining the output weights, therefore, allowing for deriving their values simultaneously. Besides, the constructed regularization function ensures the convexity of the model, which contributes to a globally optimal solution. The convergence analysis of the iterative algorithm guarantees the effectiveness of the model training. Experimental results on some UCI benchmarks and the Yale face database B indicate the superiority of our proposed algorithm.

Keywords

• Adaptive
• convergence
• convexity
• extreme learning machine (ELM)
• regularization