Frontiers in Big Data (Jan 2022)
Preparing Distributed Computing Operations for the HL-LHC Era With Operational Intelligence
- Alessandro Di Girolamo,
- Federica Legger,
- Panos Paparrigopoulos,
- Jaroslava Schovancová,
- Thomas Beermann,
- Michael Boehler,
- Daniele Bonacorsi,
- Daniele Bonacorsi,
- Luca Clissa,
- Luca Clissa,
- Leticia Decker de Sousa,
- Leticia Decker de Sousa,
- Tommaso Diotalevi,
- Tommaso Diotalevi,
- Luca Giommi,
- Luca Giommi,
- Maria Grigorieva,
- Domenico Giordano,
- David Hohn,
- Tomáš Javůrek,
- Stephane Jezequel,
- Valentin Kuznetsov,
- Mario Lassnig,
- Vasilis Mageirakos,
- Micol Olocco,
- Siarhei Padolski,
- Matteo Paltenghi,
- Lorenzo Rinaldi,
- Lorenzo Rinaldi,
- Mayank Sharma,
- Simone Rossi Tisbeni,
- Nikodemas Tuckus
Affiliations
- Alessandro Di Girolamo
- CERN, Geneva, Switzerland
- Federica Legger
- INFN Turin, Torino, Italy
- Panos Paparrigopoulos
- CERN, Geneva, Switzerland
- Jaroslava Schovancová
- CERN, Geneva, Switzerland
- Thomas Beermann
- Bergische Universitaet Wuppertal, Wuppertal, Germany
- Michael Boehler
- Physikalisches Institut, Albert-Ludwigs-Universitaet Freiburg, Freiburg, Germany
- Daniele Bonacorsi
- University of Bologna, Bologna, Italy
- Daniele Bonacorsi
- INFN Bologna, Bologna, Italy
- Luca Clissa
- University of Bologna, Bologna, Italy
- Luca Clissa
- INFN Bologna, Bologna, Italy
- Leticia Decker de Sousa
- University of Bologna, Bologna, Italy
- Leticia Decker de Sousa
- INFN Bologna, Bologna, Italy
- Tommaso Diotalevi
- University of Bologna, Bologna, Italy
- Tommaso Diotalevi
- INFN Bologna, Bologna, Italy
- Luca Giommi
- University of Bologna, Bologna, Italy
- Luca Giommi
- INFN Bologna, Bologna, Italy
- Maria Grigorieva
- Lomonosov Moscow State University, Moscow, Russia
- Domenico Giordano
- CERN, Geneva, Switzerland
- David Hohn
- Physikalisches Institut, Albert-Ludwigs-Universitaet Freiburg, Freiburg, Germany
- Tomáš Javůrek
- CERN, Geneva, Switzerland
- Stephane Jezequel
- LAPP, Université Grenoble Alpes, Univrsité. Savoie Mont Blanc, CNRS/IN2P3, Annecy, France
- Valentin Kuznetsov
- Cornell University, Ithaca, NY, United States
- Mario Lassnig
- CERN, Geneva, Switzerland
- Vasilis Mageirakos
- CERN, Geneva, Switzerland
- Micol Olocco
- INFN Turin, Torino, Italy
- Siarhei Padolski
- 0Brookhaven National Laboratory, Upton, NY, United States
- Matteo Paltenghi
- CERN, Geneva, Switzerland
- Lorenzo Rinaldi
- University of Bologna, Bologna, Italy
- Lorenzo Rinaldi
- INFN Bologna, Bologna, Italy
- Mayank Sharma
- CERN, Geneva, Switzerland
- Simone Rossi Tisbeni
- 1INFN-CNAF Bologna, Bologna, Italy
- Nikodemas Tuckus
- 2Vilnius University, Vilnius, Lithuania
- DOI
- https://doi.org/10.3389/fdata.2021.753409
- Journal volume & issue
-
Vol. 4
Abstract
As a joint effort from various communities involved in the Worldwide LHC Computing Grid, the Operational Intelligence project aims at increasing the level of automation in computing operations and reducing human interventions. The distributed computing systems currently deployed by the LHC experiments have proven to be mature and capable of meeting the experimental goals, by allowing timely delivery of scientific results. However, a substantial number of interventions from software developers, shifters, and operational teams is needed to efficiently manage such heterogenous infrastructures. Under the scope of the Operational Intelligence project, experts from several areas have gathered to propose and work on “smart” solutions. Machine learning, data mining, log analysis, and anomaly detection are only some of the tools we have evaluated for our use cases. In this community study contribution, we report on the development of a suite of operational intelligence services to cover various use cases: workload management, data management, and site operations.
Keywords
