On Adapting the DIET Architecture and the Rasa Conversational Toolkit for the Sentiment Analysis Task

Abstract

The Rasa open-source toolkit provides a valuable Natural Language Understanding (NLU) infrastructure to assist the development of conversational agents. In this paper, we show that this infrastructure can seamlessly and effectively be used for other different NLU-related text classification tasks, such as sentiment analysis. The approach is evaluated on three widely used datasets containing movie reviews, namely IMDb, Movie Review (MR) and the Stanford Sentiment Treebank (SST2). The results are consistent across the three databases, and show that even simple configurations of the NLU pipeline lead to accuracy rates that are comparable to those obtained with other state-of-the-art architectures. The best results were obtained when the Dual Intent and Entity Transformer (DIET) architecture was fed with pre-trained word embeddings, surpassing other recent proposals in the sentiment analysis field. In particular, accuracy rates of 0.907, 0.816 and 0.858 were obtained for the IMDb, MR and SST2 datasets, respectively.

Keywords

• Sentiment analysis
• Rasa
• DIET
• sentence classification
• NLU