A polymorphic cooperative compression strategy for IoT time series data based on NVM

Abstract

The compression strategy plays an important role in the performance of IoT time series data storage system.However, the current compression strategies can not adapt to the characteristics of NVM and IoT time series data.This paper proposes a polymorphic cooperative compression strategy for IoT time-series data based on NVM.Firstly, the overall structure of IoT time series data is given.Then, to address the consistent patterns in IoT time series data and the different granularity between user-space and kernel-space operations on NVM, a dual-compression strategy is devised.Initially, a lightweight compression method is applied directly as IoT time series data is received in user-space.This method efficiently reduces the volume of data for storage, while minimizing the impact on the timeliness of data storage.Moreover, a deep compression algorithm is designed for the kernel-space, primarily focusing on querying and analyzing anomalous time series data.Additionally, to address the competition for NVM bandwidth between deep compression and data storage, a dynamic adjustment algorithm that guarantees write bandwidth is proposed.Finally, a prototype of the polymorphic cooperative compression strategy is implemented and YCSB-TS is used to evaluate the results.The results show that the proposed method can effectively improve the write throughput of IoT time-series data by up to 161.3% and reduce the storage space by up to 14.6%, compared with InfluxDB, OpenTSDB, KairosDB and TVStore.

Keywords

• data compression;IoT;time series data;non-volatile memory;storage system