Enhanced Sentiment Analysis and Topic Modeling During the Pandemic Using Automated Latent Dirichlet Allocation

Abstract

The COVID-19 pandemic has profoundly impacted human societies, resulting in the loss of millions of lives and slowing economic growth worldwide. This devastating pandemic underscores the gravity of viral threats and led to multifaceted consequences, including loss of livelihoods, dynamic labor force migration, and significant ramifications on mental health. Furthermore, different scientific institutions and companies are attempting to accelerate research and innovation by analyzing large data corpus for fighting against the pandemic. In this research study, an advanced approach based on automated Latent Dirichlet Allocation (LDA) is suggested dealing with a large data corpus for efficiently providing visualization of sentiment analysis and discovered topics. This innovative approach seeks to interrogate a substantial pandemic corpus, delving into the intricacies of public sentiment and discerning evolving trends pertinent to the pandemic. A sophisticated 10-topic LDA model was implemented, revealing Topic 8 as the most prevalent, with a frequency peak of 22.29, eclipsing other enumerated topics. We employ text-mining techniques like WordCloud and Word2Vec to offer insights into specific terms relevant to the pandemic, such as “Origin,” “Symptom,” “Diagnostic,” and “Transmission.” Applying the t-SNE method enriches the analysis by visually unraveling semantic clusters within the corpus. The subsequent phase involves modeling strategic topics within the corpus through an unsupervised LDA-based approach, leveraging our suggested framework. This novel perspective contributes to a deeper understanding of the underlying dynamics by analyzing a large data corpus quickly and automatically for providing visualization of discovered topics aiming to aid front-line workers, healthcare practitioners, and community support to fight against the pandemic.

Keywords

• Topic modeling
• LDA
• sentiment analysis
• machine learning
• deep learning
• feature extraction