Progressive Detection of Uncertainty in Natural Language Processing Using a Labeled Variable Dimension Kalman Filter

Abstract

Owing to rapid advancements in information and communication technology, natural language processing (NLP) methods have attracted considerable attention. Estimating uncertainty is a well-established problem in NLP. Developing systems capable of automatically detecting uncertain terms within a natural language is critical for developing effective text-analysis algorithms. Consequently, several NLP tools have been developed for this purpose. However, there are several obstacles to the design of fast and effective NLP technologies that effectively analyze ambiguous terms in natural languages. This work focuses on a particular method for detecting uncertainty, termed progressive detection, which can be accomplished using a dynamic probabilistic graphical model to solve these challenges. We iteratively upgraded the Kalman Filter to Labeled Variable Dimension Kalman Filter (LVDKF) based on observations from real datasets. The LVDKF learns two basic features from positive and negative words. The formula was then derived using a dynamic detection system, designed based on conditional probability. Finally, we simulated progressive detection experimental conditions and assessed the LVDKF on two publicly available datasets. It performs better than the baseline methods in these tests, which shows that it is effective at detecting changes over time.