Emotion detection of social data: APIs comparative study

Abstract

The development of emotion detection technology has emerged as an efficient possibility in the corporate sector due to the nearly limitless uses of this new discipline, particularly with the unceasing propagation of social data. In recent years, the electronic marketplace has witnessed the establishment of various start-up businesses with an almost sole focus on building new commercial and open-source tools and APIs for emotion detection and recognition. Yet, these tools and APIs must be continuously reviewed and evaluated, and their performances should be reported and discussed. There is a lack of research to empirically compare current emotion detection technologies in terms of the results obtained from each model using the same textual dataset. Also, there is a lack of comparative studies that apply benchmark comparisons to social data. This study compares eight technologies: IBM Watson Natural Language Understanding, ParallelDots, Symanto – Ekman, Crystalfeel, Text to Emotion, Senpy, Textprobe, and Natural Language Processing Cloud. The comparison was undertaken using two different datasets. The emotions from the chosen datasets were then derived using the incorporated APIs. The performance of these APIs was assessed using the aggregated scores they delivered and the theoretically proven evaluation metrics such as the micro-average of accuracy, classification error, precision, recall, and f1-score. Lastly, the assessment of these APIs incorporating the evaluation measures is reported and discussed.

Keywords

• Emotion detection
• Emotion analysis
• Application programming interfaces
• Social emotion analysis
• Commercial tools
• Comparative study