Design of coal preparation data center platform based on Hadoop ecosystem

Abstract

The existing coal preparation plant information management system uses nonstandard interface, which leads to repeated data collection, and each system is independent of each other, and the capability to process multi-source heterogeneous data is weak. In order to solve above problems, based on big data technology of Hadoop ecosystem, a coal preparation data center platform design scheme based on Hadoop ecosystem is proposed. The system integration is realized by defining data standards through master data management system and enterprise service bus. Normalization, correlation coefficient matrix and noise abnormal point detection programs are designed to realize data processing. DS (Dempster-Shafer) evidence theory, Hadoop and Hive data warehouse are combined to design multi-source heterogeneous data fusion subsystem to realize data fusion. Highcharts data visualization components are used to achieve interactive visualization of data. The practical application results show that the data center platform realizes the standardization of master data definition standard and system integration interface, improves the processing capability of coal preparation data, realizes the fusion and sharing of multi-source heterogeneous coal preparation data, and realizes the real-time interactive visualization of data.

Keywords

• intelligent coal preparation plant; hadoop ecosystem; data cente platform; big data; multi-source heterogeneous data; data fusion; d-s evidence theory