Distributed Framework of Artificial Neural Network for Big Data Analysis

Abstract

In this research, we introduce a distributed framework of artificial neural network (ANN) to deal with the big data real‐time analysis and return proper outcomes in very short delay. The result of our experiment shows that training the distributed ANN model could be converged in 17 seconds on 24‐core clustering platform and learns that multi‐model with stratification strategy would obtain most true positive predictions with nearly 70% precision at voting threshold value equal to 0.7.In our system, ANNs are used in the data mining process for identifying patterns in financial time series. We implement a framework for training ANNs on a distributed computing platform. We adopt Apache Spark to build the base computing cluster because it is capable of high performance in‐memory computing. We investigate a number of distributed back propagation algorithms and techniques, especially ones for time series prediction, and incorporate them into our framework with some modifications. With various options for the details, we provide the user with flexibility in neural network modeling.

Keywords

• artificial neural network
• big data analysis
• data mining
• distributed computing
• multilayer perceptron