Out-of-context misinformation detection method based on stance analysis

Abstract

As artificial intelligence-based automatic generation technology advances, the emergence of misinformation in more sophisticated guises has become a significant threat to the economy and social order. Among its forms, out-of-context misinformation stands out as particularly deceptive and readily executable. This type of misinformation involves malicious actors enhancing the credibility of false narratives by misrepresenting contextual details such as individuals, events, and locations within real images. To address the shortcomings of current detection algorithms, which heavily depend on knowledge bases and often overlook the stance relationship between the information under scrutiny and online evidence, a stance analysis-based out-of-context misinformation detection method was developed. This method involved several steps for the detection of an image-caption pair along with the corresponding textual and visual evidence retrieved from the Internet. Initially, a stance gain score for each piece of textual evidence was calculated based on the co-occurrence of named entities. Subsequently, independent stance analysis networks were utilized to perform hierarchical clustering on both the image and visual evidence, as well as on the caption and textual evidence. This process involved the extraction of semantic stance representations, facilitated by multiple attention mechanisms and a stance analysis module. The authenticity of the image-caption pair was subsequently predicted based on the outcomes of semantic comparison and stance analysis. Experimental results indicate that the incorporation of stance analysis significantly enhances the method's detection capabilities. Specifically, the accuracy of this method outperforms the state-of-the-art algorithm that employs Internet evidence for detection by 2.3%.

Keywords

• out-of-context
• stance analysis
• misinformation
• internet evidence