An online health community user intention identification method based on BERT-BiGRU-Attention

Abstract

Aiming at the problem of high cost and low expansibility of traditional user intention recognition, which mainly uses template matching or artificial feature set, a hybrid neural network intention recognition model based on BERT word embedding and BiGRU-Attention was proposed. First, the word embedding pre-trained by BERT was used as the input, and the features of the interrogative sentences were extracted by BiGRU. Then, the attention mechanism was introduced to extract the information of words that have important influence on the meaning of sentences and allocate the corresponding weights, so as to obtain the sentence embedding that integrates the word-level weights and input it into the softmax classifier to realize intention classification. According to the experiment on the crawling corpus, it shows that the performance of BERT-BiGRU-Attention method is better than that of traditional template matching, SVM and lately popular CNN-LSTM deep learning combined model. The proposed method can effectively improve the performance of intention recognition model and the quality of online health information service, which provide technical support for the online health community question answering system.

Keywords

• natural language processing; intention identification; online health communities; bert word embedding; bidirectional gated recurrent unit(bigru); attention mechanism