IEEE Access (Jan 2020)
A Cloud-Based Framework for Machine Learning Workloads and Applications
- Alvaro Lopez Garcia,
- Jesus Marco De Lucas,
- Marica Antonacci,
- Wolfgang Zu Castell,
- Mario David,
- Marcus Hardt,
- Lara Lloret Iglesias,
- Germen Molto,
- Marcin Plociennik,
- Viet Tran,
- Andy S. Alic,
- Miguel Caballer,
- Isabel Campos Plasencia,
- Alessandro Costantini,
- Stefan Dlugolinsky,
- Doina Cristina Duma,
- Giacinto Donvito,
- Jorge Gomes,
- Ignacio Heredia Cacha,
- Keiichi Ito,
- Valentin Y. Kozlov,
- Giang Nguyen,
- Pablo Orviz Fernandez,
- Zdenek Sustr,
- Pawel Wolniewicz
Affiliations
- Alvaro Lopez Garcia
- ORCiD
- IFCA (CSIC-UC), Santander, Spain
- Jesus Marco De Lucas
- ORCiD
- IFCA (CSIC-UC), Santander, Spain
- Marica Antonacci
- ORCiD
- INFN Bari, Bari, Italy
- Wolfgang Zu Castell
- ORCiD
- Helmholtz Zentrum München, Deutsches Forschungszentrum für Gesundheit, Oberschleißheim, Germany
- Mario David
- ORCiD
- Laboratory of Instrumentation and Experimental Particle Physics, Lisbon, Portugal
- Marcus Hardt
- ORCiD
- Karlsruhe Institute of Technology, Karlsruhe, Germany
- Lara Lloret Iglesias
- ORCiD
- IFCA (CSIC-UC), Santander, Spain
- Germen Molto
- ORCiD
- Instituto de Instrumentación para Imagen Molecular (I3M), CSIC, Universitat Politècnica de València, Valencia, Spain
- Marcin Plociennik
- ORCiD
- Poznan Supercomputing and Networking Center, IBCh PAS, Poznan, Poland
- Viet Tran
- ORCiD
- Slovak Academy of Sciences (IISAS), Institute of Informatics, Bratislava, Slovakia
- Andy S. Alic
- ORCiD
- Instituto de Instrumentación para Imagen Molecular (I3M), CSIC, Universitat Politècnica de València, Valencia, Spain
- Miguel Caballer
- ORCiD
- Instituto de Instrumentación para Imagen Molecular (I3M), CSIC, Universitat Politècnica de València, Valencia, Spain
- Isabel Campos Plasencia
- ORCiD
- IFCA (CSIC-UC), Santander, Spain
- Alessandro Costantini
- ORCiD
- INFN CNAF, Bologna, Italy
- Stefan Dlugolinsky
- ORCiD
- Slovak Academy of Sciences (IISAS), Institute of Informatics, Bratislava, Slovakia
- Doina Cristina Duma
- ORCiD
- INFN CNAF, Bologna, Italy
- Giacinto Donvito
- ORCiD
- INFN Bari, Bari, Italy
- Jorge Gomes
- ORCiD
- Laboratory of Instrumentation and Experimental Particle Physics, Lisbon, Portugal
- Ignacio Heredia Cacha
- ORCiD
- IFCA (CSIC-UC), Santander, Spain
- Keiichi Ito
- ORCiD
- Helmholtz Zentrum München, Deutsches Forschungszentrum für Gesundheit, Oberschleißheim, Germany
- Valentin Y. Kozlov
- ORCiD
- Karlsruhe Institute of Technology, Karlsruhe, Germany
- Giang Nguyen
- ORCiD
- Slovak Academy of Sciences (IISAS), Institute of Informatics, Bratislava, Slovakia
- Pablo Orviz Fernandez
- ORCiD
- IFCA (CSIC-UC), Santander, Spain
- Zdenek Sustr
- ORCiD
- CESNET, Prague, Czech Republic
- Pawel Wolniewicz
- ORCiD
- Poznan Supercomputing and Networking Center, IBCh PAS, Poznan, Poland
- DOI
- https://doi.org/10.1109/ACCESS.2020.2964386
- Journal volume & issue
-
Vol. 8
pp. 18681 – 18692
Abstract
In this paper we propose a distributed architecture to provide machine learning practitioners with a set of tools and cloud services that cover the whole machine learning development cycle: ranging from the models creation, training, validation and testing to the models serving as a service, sharing and publication. In such respect, the DEEP-Hybrid-DataCloud framework allows transparent access to existing e-Infrastructures, effectively exploiting distributed resources for the most compute-intensive tasks coming from the machine learning development cycle. Moreover, it provides scientists with a set of Cloud-oriented services to make their models publicly available, by adopting a serverless architecture and a DevOps approach, allowing an easy share, publish and deploy of the developed models.
Keywords
- Cloud computing
- computers and information processing
- deep learning
- distributed computing
- machine learning
- serverless architectures