Integrating PubMed Label Hierarchy Knowledge into a Complex Hierarchical Deep Neural Network

Abstract

This paper proposes an innovative method that exploits a complex deep learning network architecture, called Hierarchical Deep Neural Network (HDNN), specifically developed for the eXtreme Multilabel Text Classification (XMTC) task, when the label set is hierarchically organized, such as the case of the PubMed article labeling task. In detail, the topology of the proposed HDNN architecture follows the exact hierarchical structure of the label set to integrate this knowledge directly into the DNN. We assumed that if a label set hierarchy is available, as in the case of the PubMed Dataset, forcing this information into the network topology could enhance the classification performances and the interpretability of the results, especially related to the hierarchy. We performed an experimental assessment of the PubMed article classification task, demonstrating that the proposed HDNN provides performance improvement for a baseline based on a classic flat Convolution Neural Network (CNN) deep learning architecture, in particular in terms of hierarchical measures. These results provide useful hints for integrating previous and innate knowledge in a deep neural network. The drawback of the HDNN is the high computational time required to train the neural network, which can be addressed with a parallel implementation planned as a future work.

Keywords

• extreme multilabel text classification
• hierarchical deep neural network
• natural language processing
• BioBERT
• PubMed MeSH