Dynamic heatmap pyramid computation for massive high-parallel spatial streaming in urban environments

Abstract

It is necessary to integrate data and information effectively in order to build an integrated digital and refined urban system. A multi-granularity and multi-view urban heat map formed by integrating multi-source urban information flow can assist in immediate decision-making. Managing urban big data in the form of streams requires a stable operating environment that can efficiently handle millions of sensors and devices connected to the Internet of Things (IoT) in a highly concurrent environment. Urban management data contain spatiotemporal multi-dimensional information that presents the complexity of spatiotemporal dynamic associations, further increasing the difficulty of data streaming. To overcome these challenges, we have proposed a spatiotemporal-pyramid (ST-pyramid) model that organizes multidimensional and dynamic data streams logically using a data partition strategy based on geographical grid subdivision. Accordingly, we have proposed a load-balanced heatmap pyramid computation framework that can be used to build streaming processing procedures in a distributed environment. In addition, we implemented an urban traffic heatmap prototype system based on an open-source Flink framework. The experimental results show that the real-time heatmap pyramid construction algorithm proposed in this paper has high throughput, low latency, and flexible scalability and can provide large-scale public services in time in digital earth construction.

Keywords

• Big data
• stream processing
• urban system
• distributed computing
• heatmap