GAP: Genetic Algorithm Based Large-Scale Graph Partition in Heterogeneous Cluster

Abstract

Graph is an important model to describe various networks, and its scale becomes larger and larger with the development of communication and information technology. The analysis of large-scale graphs requires distributed graph processing systems, and graph partition is the basis of these systems. The existing graph partitioning algorithms are almost proposed for homogeneous clusters, which don't consider the differences among computing nodes in heterogeneous clusters. This paper proposes GAP, a Genetic Algorithm based graph Partitioning algorithm to solve this problem. GAP aims to reduce the total processing time on a heterogeneous cluster by partitioning graphs according to the computing powers of computing nodes. GAP balanced partition the graph initially, and then utilizes genetic algorithm to transfer vertices to reduce cut edges. GAP can balance the processing time of computing nodes, and reduce the communication time among computing nodes. The experiments performed on a heterogeneous cluster demonstrate the outperformance of GAP than Hash.

Keywords

• Distributed graph processing system
• graph partition
• genetic algorithm
• heterogeneous cluster