An Eclectic Approach for Enhancing Language Models Through Rich Embedding Features

Abstract

Text processing is a fundamental aspect of Natural Language Processing (NLP) and is crucial for various applications in fields such as artificial intelligence, data science, and information retrieval. It plays a core role in language models. Most text-processing approaches focus on describing and synthesizing, to a greater or lesser degree, lexical, syntactic, and semantic properties of text in the form of numerical vectors that induce a metric space, in which, it is possible to find underlying patterns and structures related to the original text. Since each approach has strengths and weaknesses, finding a single approach that perfectly extracts representative text properties for every task and application domain is hard. This paper proposes a novel approach capable of synthesizing information from heterogeneous state-of-the-art text processing approaches into a unified representation. Encouraging results demonstrate that using this representation in popular machine-learning tasks not only leads to superior performance but also offers notable advantages in memory efficiency and preservation of underlying information of the distinct sources involved in such a representation.

Keywords

• Self-organizing map
• word embeddings
• feature extraction
• natural language processing