The Cherenkov Telescope Array Observatory workflow management system

Abstract

The Cherenkov Telescope Array Observatory (CTAO) is the next generation ground-based observatory for gamma-ray astronomy at very high energies. It is expected to produce about 2 PB of raw data each year and to manage a global data volume which will grow through the years to reach about 100 PB in 2030. In addition, CTAO will require a high computing capacity for data processing and Monte Carlo simulations, of the order of hundreds of millions of CPU HS06 hours per year. To meet these requirements, CTAO will adopt a distributed computing model using 4 academic data centers, and will use the DIRAC framework as its workload management system. In the past ten years, to optimize the instrument design and study its performances, CTAO has used the European Grid Infrastructure (EGI) to run massive Monte Carlo campaigns. In order to handle these campaigns and to automatize simulation and data processing workflows, we have developed a production system prototype based on DIRAC. Recently, we have also developed a user interface allowing for the configuration and submission of complex workflows. In this contribution we present the production system prototype, its user interface for workflow management as well as its application to CTAO workflows.