IPFSz: An Efficient Data Compression Scheme in InterPlanetary File System

Abstract

InterPlanetary File System (IPFS) is a peer-to-peer (P2P) distributed file system that allows data to be shared and exchanged between connected nodes. Recently, IPFS aims to replace existing protocols used in a centralized system (e.g., HTTP) by overcoming their weaknesses such as a single point of failure and arbitrary control. However, we observe that the performance of IPFS is worse as the data size increases since it demands large numbers of storage and network I/O operations. In this paper, we present an efficient data transmission scheme in IPFS called IPFSz to improve the I/O performance of IPFS. To do this, we extend IPFS by enabling real-time compression functionality while maintaining its existing operations in IPFS. In IPFSz, we compress/decompress the data during I/O operations by using a compression algorithm, manage the states of data (e.g., compressed or decompressed), and provide a compression/decompression interface to applications. Thus, IPFSz can reduce the number of storage and network I/O operations and storage space without sacrificing the integrity of the existing IPFS. We have implemented and evaluated IPFSz on five Amazon EC2 nodes. The experimental results show that IPFSz provides higher performance by up to $7.9\times $ and saves the storage space by up to $6.3\times $ compared with existing IPFS.

Keywords

• InterPlanetary file system
• data compression
• decentralized system
• P2P network
• distributed file system