MsCoa: Multi-Step Co-Attention Model for Multi-Label Classification

Abstract

Multi-label text classification (MLC) task, as one of the sub-tasks of natural language processing, has broad application prospects. On the basis of studying the previous research work, this research takes the relationship among text information, leading label information and predictive label information as the frame and analyzes the information loss of original text and leading label, decoding error accumulation. We propose an improved multi-step multi-classification model to mitigate the phenomenon of error prediction, label repetition and error accumulation. The model uses multi-step and multi-classification task to complete multi-label prediction. It uses the leading label and the original text as input, and the next to-be-predicted label as output. The co-attention mechanism is operated between the original text and the leading label. The attention of the original text to the leading label is helpful to filter out the error accumulation problem caused by the error prediction. The features is combined in a manner of difference and concatenation, which highlights the auxiliary effect of the model structure on feature extraction. In order to avoid the influence of the feature dimension on the performance of long short-term memory (LSTM), a multi-layer fully-connected classifier is used instead to predict the label. Through experimental validation, the performance of our model on the multi-label text classification task shows the current optimal level, which fully proves the superiority of our model.

Keywords

• Artificial neural network
• attention mechanism
• deep learning
• multi-label classification