A Dynamic Distributed Deterministic Load-Balancer for Decentralized Hierarchical Infrastructures

Abstract

In this work, we propose D3-Tree, a dynamic distributed deterministic structure for data management in decentralized networks, by engineering and extending an existing decentralized structure. Conducting an extensive experimental study, we verify that the implemented structure outperforms other well-known hierarchical tree-based structures since it provides better complexities regarding load-balancing operations. More specifically, the structure achieves an O(logN) amortized bound (N is the number of nodes present in the network), using an efficient deterministic load-balancing mechanism, which is general enough to be applied to other hierarchical tree-based structures. Moreover, our structure achieves O(logN) worst-case search performance. Last but not least, we investigate the structure’s fault tolerance, which hasn’t been sufficiently tackled in previous work, both theoretically and through rigorous experimentation. We prove that D3-Tree is highly fault-tolerant and achieves O(logN) amortized search cost under massive node failures, accompanied by a significant success rate. Afterwards, by incorporating this novel balancing scheme into the ART (Autonomous Range Tree) structure, we go one step further to achieve sub-logarithmic complexity and propose the ART+ structure. ART+ achieves an O(logb2logN) communication cost for query and update operations (b is a double-exponentially power of 2 and N is the total number of nodes). Moreover, ART+ is a fully dynamic and fault-tolerant structure, which supports the join/leave node operations in O(loglogN) expected WHP (with high proability) number of hops and performs load-balancing in O(loglogN) amortized cost.

Keywords

• decentralized systems
• distributed data structures
• P2P
• load balancing
• fault tolerance