A Two-Layer Architecture for Failure Prediction Based on High-Dimension Monitoring Sequences

Abstract

In recent years, the distributed architecture has been widely adopted by security companies with the rapid expansion of their business. A distributed system is comprised of many computing nodes of different components which are connected by high-speed communication networks. With the increasing functionality and complexity of the systems, failures of nodes are inevitable which may result in considerable loss. In order to identify anomalies of the possible failures and enable DevOps engineers to operate in advance, this paper proposes a two-layer prediction architecture based on the monitoring sequences of nodes status. Generally speaking, in the first layer, we make use of EXPoSE anomaly detection technique to derive anomaly scores in constant time which are then used as input data for ensemble learning in the second layer. Experiments are conducted on the data provided by one of the largest security companies, and the results demonstrate the predictability of the proposed approach.