Triple Channel Feature Fusion Few-Shot Intent Recognition With Orthogonality Constrained Multi-Head Attention

Abstract

Intent recognition in few-shot scenarios is a hot research topic in natural language understanding tasks. Aiming at the problems of insufficient consideration of fine-grained features of the text and insufficient training of features in the process of model fine-tuning, the Triple Channel IntentBERT and Orthogonality Constrained Multi-Head Attention Model (TMH-IntentBERT) is proposed. The part-of-speech features, word features and keyword features are combined to extract fine-grained features of data. And the a priori knowledge of the text is fully utilized. Context information is captured through multi-head attention to learn diversified representations. At the same time, the context and score vector regularization terms are added to reduce the position and representation redundancy between heads and enhance the diversity. The experimental results show that on the public dataset, the TMH-IntentBERT model has a minimum increase of 0.63%, 0.73%, 0.79%, and 1.10% in accuracy, precision, F1 value and AUROC compared with CONVBERT, TOD-BERT, WikiHowRoBERTA, IntentBERT and DFT++, respectively.

Keywords

• Intention recognition
• few-shot
• feature fusion
• multi-head attention
• IntentBERT