An Approach Based on Semantic Relationship Embeddings for Text Classification

Abstract

Semantic relationships between words provide relevant information about the whole idea in the texts. Existing embedding representation models characterize each word as a vector of numbers with a fixed length. These models have been used in tasks involving text classification, such as recommendation and question–answer systems. However, the embedded information provided by semantic relationships has been neglected. Therefore, this paper proposes an approach that involves semantic relationships in embedding models for text classification, which is evaluated. Three embedding models based on semantic relations extracted from Wikipedia are presented and compared with existing word-based models. Our approach considers the following relationships: synonymy, hyponymy, and hyperonymy. They were considered since previous experiments have shown that they provide semantic knowledge. The relationships are extracted from Wikipedia using lexical-syntactic patterns identified in the literature. The extracted relationships are embedded as a vector: synonymy, hyponymy–hyperonymy, and a combination of all relationships. A Convolutional Neural Network using semantic relationship embeddings was trained for text classification. An evaluation was carried out for the proposed relationship embedding configurations and existing word-based models to compare them based on two corpora. The results were obtained with the metrics of precision, accuracy, recall, and F1-measure. The best results for the 20-Newsgroup corpus were obtained with the hyponymy–hyperonymy embeddings, achieving an accuracy of 0.79. For the Reuters corpus, F1-measure and recall of 0.87 were obtained using synonymy–hyponymy–hyperonymy.

Keywords

• deep learning
• semantic relationship embeddings
• lexical syntactic patterns
• convolutional neural networks
• text classification