Diverse Models, United Goal: A Comprehensive Survey of Ensemble Learning

Abstract

ABSTRACT Ensemble learning, a pivotal branch of machine learning, amalgamates multiple base models to enhance the overarching performance of predictive models, capitalising on the diversity and collective wisdom of the ensemble to surpass individual models and mitigate overfitting. In this review, a four‐layer research framework is established for the research of ensemble learning, which can offer a comprehensive and structured review of ensemble learning from bottom to top. Firstly, this survey commences by introducing fundamental ensemble learning techniques, including bagging, boosting, and stacking, while also exploring the ensemble's diversity. Then, deep ensemble learning and semi‐supervised ensemble learning are studied in detail. Furthermore, the utilisation of ensemble learning techniques to navigate challenging datasets, such as imbalanced and high‐dimensional data, is discussed. The application of ensemble learning techniques across various research domains, including healthcare, transportation, finance, manufacturing, and the Internet, is also examined. The survey concludes by discussing challenges intrinsic to ensemble learning.

Keywords

• bagging
• boosting
• deep learning
• ensemble learning
• imbalanced data
• semi‐supervised learning