Efficient Storage Scaling for MBR and MSR Codes

Abstract

Due to the explosive growth of storage demands, distributed storage systems need to support storage scaling efficiently. Recent work optimizes scaling in a decentralized manner for Reed-solomon coded storage systems. In this paper, we focus on storage scaling for storage systems with regenerating codes and design two efficient scaling algorithms for minimum bandwidth regenerating (MBR) and minimum storage regenerating (MSR) codes. We integrate these two scaling algorithms into Hadoop Distributed File System (HDFS), and the experiments on Amazon EC2 show that the scaling bandwidth can be reduced up to 75% and 43.8% over the centralized scaling.

Keywords

• Storage scaling
• erasure codes
• regenerating codes
• distributed file systems