Erasure Coding XRootD Object Store

Abstract

An erasure coding (EC) algorithm was implemented in XRootD client library, and an EC-enabled XRootD storage prototype was set up at SLAC for evaluation. The architecture and configuration of the prototype is almost identical to that of a traditional non-EC XRootD storage behind a firewall: a backend XRootD storage cluster in its simplest form, and an internet facing XRootD proxy. This proxy handles EC and spreads the data stripes of a file/object across several backend nodes. It also supports all functions used on a WLCG storage system: HTTP(s) and XRootD protocols, Third Party Copy, X509/VOMS/Token, etc. The cross-node EC architecture brings significant advantages in both performance and resilience: e.g. parallel data access, tolerance of downtime and hardware failure. It also pushes the preferred network-IO mode from Posix-like to object-like. This paper will describe the prototype’s architecture and its design choices, the performance in high concurrent throughputs and file/object operations, failure modes and their handling, data recovery methods, and administration.