Improvement of Parallelism Process in Distributed Data Processing

Abstract

Many sectors have been related to massive population growth, whether healthcare, industry, transport, or information systems. Many of these industries daily generate a vast amount of data, and a basic system of the data processing stopped to fulfill efficiencies of data manipulation. The central data processing also has various disadvantages, such as central storing of the server, low storability, and high costs. Because of these, but also another reason, the system called distributed data processing was created. Distributed data processing led to calculations acceleration, higher redundancy, and bigger storability. The massive use of the calculation servers happens during distributed data processing with big data, causing unsatisfactory performance. Based on observations, we created an architecture capable of effectively adding and deleting the calculation units based on demand in the data processing and increasing resistance against an error. The experiments show that the newly created architecture can increase security, not only of the data processing but also of processed data security. On the other hand, adaptation to the performance plays a critical role we achieved after performing the planned experiments.

Keywords

• distributed data processing
• parallelism
• scaling policy