Hierarchical Graph Transformer-Based Deep Learning Model for Large-Scale Multi-Label Text Classification

Abstract

Traditional methods of multi-label text classification, particularly deep learning, have achieved remarkable results. However, most of these methods use word2vec technology to represent sequential text information, while ignoring the logic and internal hierarchy of the text itself. Although these approaches can learn the hypothetical hierarchy and logic of the text, it is unexplained. In addition, the traditional approach treats labels as independent individuals and ignores the relationships between them, which not only does not reflect reality but also causes significant loss of semantic information. In this paper, we propose a novel Hierarchical Graph Transformer based deep learning model for large-scale multi-label text classification. We first model the text into a graph structure that can embody the different semantics of the text and the connections between them. We then use a multi-layer transformer structure with a multi-head attention mechanism at the word, sentence, and graph levels to fully capture the features of the text and observe the importance of the separate parts. Finally, we use the hierarchical relationship of the labels to generate the representation of the labels, and design a weighted loss function based on the semantic distances of the labels. Extensive experiments conducted on three benchmark datasets demonstrated that the proposed model can realistically capture the hierarchy and logic of text and improve performance compared with the state-of-the-art methods.

Keywords

• Multi-label text classification
• graph modeling
• graph transformer
• deep learning