A social media geolocation prediction method based on multimodal fusion

Abstract

Geographical information extracted from social media text reveals underlying spatial correlations.A geographical location prediction method for social media text based on multimodal fusion was proposed.By utilizing images associated with the text as augmented data, an integrated image-text dataset was constructed to enhance the accuracy of geographical location prediction.The multimodal fusion model employs separate channels for images and text to independently extract their respective geographical location information.Additionally, a text-image matching module was introduced to denoise the image-text pairs, effectively solving the issue of text-image misalignment.Experimental results on the Geotext dataset indicate that compared to the baseline model, the proposed method reduces the median error distance by 18.8% and the average error distance by 4.5%.

Keywords

• social media;geolocation;multimodal fusion;information mining