Enhancing Big Data Performance Through Graph Coloring-Based Locality of Reference

Abstract

Efficiency is a crucial factor when handling the retrieval and storage of data from vast amounts of records in a Big Data repository. These systems require a subset of data that can be accommodated within the combined physical memory of a cluster of servers. It becomes impractical to analyze all of the data if its size exceeds the available memory capacity. Retrieving data from virtual storage, primarily hard disks, is significantly slower compared to accessing data from main memory, resulting in increased access time and diminished performance. To address this, a proposed model aims to enhance performance by identifying the most suitable data locality structure within a big data set and reorganizing the data schema accordingly; by locality, it has been referred to as a particular access pattern. This allows transactions to be executed on data residing in the fastest memory layer, such as cache, main memory, or disk cache

Keywords

• Big Data
• Graph Representation
• Locality of References
• Performance
• Synthetic Memory