K-means Weibo topic discovery model based on feature fusion

Abstract

Aiming at the shortcomings of high-dimensional sparseness in the short text of Weibo on traditional topic detection methods, a K-means Weibo topic discovery model based on feature fusion was proposed. In order to better express the semantic information of Weibo topics in this paper, the word-pair vector model(Biterm_VSM) co-occurring in sentences is used instead of the traditional vector space model(VSM), and combined with the topic model(Latent Dirichlet Allocation，LDA) to mine the potential semantics of Weibo short text, merging features obtained from the two models, and applying K-means clustering algorithm to discover topics. The Experimental results show that compared with the traditional topic detection method, the model′s adjusted Rand index(ARI) is 0.80, which is 3%~6% higher than the traditional topic detection method.

Keywords

• topic detection
• biterm_vsm
• lda
• feature fusion
• k-means