DeepFM and Convolutional Neural Networks Ensembles for Multimodal Rumor Detection

Abstract

With the increasing popularity of social media represented by Weibo,rumors spread rapidly through social media,which is more likely to cause serious consequences.The problem of automatic rumor detection has attracted widespread attention from academic and industrial circles at home and abroad.We have noticed that more and more users use pictures to post Weibo,not just text.Weibo usually consists of text,images and social context.Therefore,a multi-modal network rumor detection method DCNN based on deep neural network for the text content,image and user attribute information of the accompanying text is proposed.This method consists of a multi-modal feature extractor and a rumor detector.The multi-modal feature extractor is divided into three parts:a text feature extractor based on TextCNN,a picture feature extractor based on VGG-19,and a user social feature extractor based on DeepFM algorithm.These three parts learn feature representations on different modalities of Weibo to form re-parameterized multi-modal features,which are fused as input to the rumor detector classification detection.This algorithm has carried out a large number of experiments on the Weibo data set,and the experimental results show that the recognition accuracy of DCNN algorithm is improved from 78.1% to 80.3%,which verifies the feasibility and effectiveness of DCNN algorithm and feature interaction method for social characteristics.

Keywords

• multimodal|rumor detection|deepfm|convolutional neural networks|social feature|natural language processing