ASIC design of LZ77 compressor for computational storage drives

Abstract

Abstract The advent of cloud computing, 5G, and artificial intelligence technology has spurred the exponential growth of big data and the need for data compression. To alleviate the CPU burden, computational engines for data compression integrated into network cards and storage devices, giving rise to computational storage. However, computational storage faces several challenges, such as limited memory resources for large hash tables, data dependency for parallel encoding and decoding, and stringent requirements for bandwidth, latency, and power efficiency. Here, an application‐specific integrated circuit‐based LZ77 solution that aims to address these challenges is proposed. Our solution reduces the hash table resource consumption and the bandwidth fluctuations, while achieving high compression ratios. The solution is implemented using Taiwan Semiconductor Manufacturing Company Limited 12 nm technology and demonstrate that a single LZ77 engine can achieve a 4 GB/s throughput at 1 GHz clock frequency. By using four engines, a computational storage drive to achieve 16 GB/s compression bandwidth is enabled. The solution achieves a comparable compression ratio as the standard LZ77 algorithm with a negligible compression ratio loss of 5%, balancing efficiency and effectiveness.

Keywords

• acceleration
• application specific integrated circuits
• data compression
• encoding
• storage management chips