WLCG Dashboards with Unified Monitoring

Abstract

Monitoring of the CERN Data Centres and the WLCG infrastructure is now largely based on the new monitoring infrastructure provided by CERN IT. This is the result of the migration from several old in-house developed monitoring tools into a common monitoring infrastructure based on open source technologies such as Collectd, Flume, Kafka, Spark, InfluxDB, Grafana and others. This new infrastructure relies on CERN IT services (OpenStack, Puppet, Gitlab, DBOD, etc) and covers the full range of monitoring tasks: metrics and logs collection, alarms generation, data validation and transport, data enrichment and aggregation (where applicable), dashboards visualisation, reports generation, etc. This contribution will present the different services offered by the infrastructure today, highlight the main monitoring use cases from the CERN Data Centres and WLCG, and analyse the last years experience of moving from legacy well-established custom monitoring tools into a common open source-based infrastructure.