Apache Hama: An Emerging Bulk Synchronous Parallel Computing Framework for Big Data Applications

Abstract

In today's highly intertwined network society, the demand for big data processing frameworks is continuously growing. The widely adopted model to process big data is parallel and distributed computing. This paper documents the significant progress achieved in the field of distributed computing frameworks, particularly Apache Hama, a top level project under the Apache Software Foundation, based on bulk synchronous parallel processing. The comparative studies and empirical evaluations performed in this paper reveal Hama's potential and efficacy in big data applications. In particular, we present a benchmark evaluation of Hama's graph package and Apache Giraph using PageRank algorithm. The results show that the performance of Hama is better than Giraph in terms of scalability and computational speed. However, despite great progress, a number of challenging issues continue to inhibit the full potential of Hama to be used at large scale. This paper also describes these challenges, analyzes solutions proposed to overcome them, and highlights research opportunities.

Keywords

• Apache Hama
• big data
• BSP
• bulk synchronous parallel
• distributed computing
• Giraph