Improving service-level agreements for critical systems using big data monitoring techniques

Abstract

The proliferation of big data in virtually every branch of society and industry comes with the need to adapt and develop monitoring and alerting systems in such a way that the system can cope with any kind of data stream, whilst also ensuring rapid response times. This paper presents a framework based on modern open-source technologies that can be used to improve the quality and reliability of a connected system (such as an industrial control system), through effective monitoring and alerting. Service level agreements are crucial in our modern society, where failures need to be detected quickly and effectively, especially when one is providing a service and every moment of downtime means a large quantity of lost money and potential customers, thus monitoring is essential. Benefits in terms of responsiveness and lower downtime are also discussed, with an emphasis on a prototype implementation for a major non-profit organization.