Dynamic load balancing scheme on massive file transfer system

Abstract

In this paper, a dynamic load balancing scheme applied to massive file transfer system is proposed. The scheme is designed to load balance FTP server cluster. Instead of recording connection number, runtime load information of each server is periodically collected and used in combination of static performance parameters collected on server startup to calculate the weight of servers. Improved Weighted Round-Robin algorithm is adopted in this scheme. Importantly, the weight of each server is initialized with static performance parameters and dynamically modified according to the runtime load. Apache Zookeeper cluster is used to receive all information and it will inform director of the runtime load variation and offline behavior of any server. In order to evaluate the effect of this scheme, a contrast experiment with LVS is also conducted.