HML-RF: Hybrid Multi-Label Random Forest

Abstract

Multi-label classification is the supervised learning problem in which an instance is associated with a set of labels. In this, labels are correlated, and hence label dependency information plays a vital role. Its always been a question of research to decide the order of labels to exploit their inter-dependency. Hence, to this end, many research works are done that, in general, can be categorized as problem transformation and algorithm adaptation techniques. The problem transformation reconstructs the multi-label problem as a multiple single class problem. The algorithm transformation modifies the existing well-known machine learning approaches to solve the multi-label classification problem. However, these two techniques have their pros and cons. In this paper, we propose a novel approach to consider the merits of both techniques, hence named Hybrid Multi-Label Random Forest (HML-RF). The multi-label decision trees are used as base classifiers in the proposed approach to construct the HML-RF model. Each base classifier is constructed over a randomly selected subset of labels to exploit the label dependency. We also formulate a way to compute the tree strength of a multi-label decision tree, which is used to construct the HML-RF with strength (HML-RFws). The efficacy of the proposed approach is tested over the ten well-known and publicly available datasets. Experimental results show the HML-RF is performing better for at-least six datasets, and the HML-RFws is performing better for at-least nine datasets in comparison to state-of-the-art approaches in terms of accuracy, hamming loss, and zero-one loss. Finally, the statistical test is also validating all the experimental results.

Keywords

• Ensemble methods
• label correlation
• multi-label classification
• machine learning
• random forest