A Spatial Information Extraction Method Based on Multi-Modal Social Media Data: A Case Study on Urban Inundation

Abstract

With the proliferation and development of social media platforms, social media data have become an important source for acquiring spatiotemporal information on various urban events. Providing accurate spatiotemporal information for events contributes to enhancing the capabilities of urban management and emergency responses. However, existing research regarding mining spatiotemporal information of events often solely focuses on textual content and neglects data from other modalities such as images and videos. Therefore, this study proposes an innovative spatiotemporal information extraction method, which extracts the spatiotemporal information of events from multimodal data on Weibo at coarse- and fine-grained hierarchical levels and serves as a beneficial supplement to existing urban event monitoring methods. This paper utilizes the “20 July 2021 Zhengzhou Heavy Rainfall” incident as an example to evaluate and analyze the effectiveness of the proposed method. Results indicate that in coarse-grained spatial information extraction using only textual data, our method achieved a spatial precision of 87.54% within a 60 m range and reached 100% spatial precision for ranges beyond 200 m. For fine-grained spatial information extraction, the introduction of other modal data, such as images and videos, resulted in a significant improvement in spatial error. These results demonstrate the ability of the MIST-SMMD (Method of Identifying Spatiotemporal Information of Social Media Multimodal Data) to extract spatiotemporal information from urban events at both coarse and fine levels and confirm the significant advantages of multimodal data in enhancing the precision of spatial information extraction.

Keywords

• multimodal
• social media
• spatiotemporal information extraction
• inundation