Experience of OpenStack Test Deployment and Comparison of Virtual and Real Cluster Environments

Abstract

Introduction. A brief overview of the properties and architecture of one of the components of the National Cloud of Open Science prototype – the cloud platform OpenStack is given. The list of software and hardware components of the OpenStack test cloud environment and the sequence of actions required for the deployment of both OpenStack itself and the Slurm virtual cluster environment for portable, scalable, reproducible scientific biomedical computing are presented. The purpose of the paper is a description of the experience of test deployment of OpenStack to create a scalable computing environment for reproducible scientific computing using modern technological solutions, which can be applied to both cloud (OpenStack, AWS, Google) and cluster platforms (Slurm). Results. The structure of the created test containerized (using Singularity technology) biomedical application, which contains modern software and libraries and can be used in conventional and cloud virtual cluster environments is briefly described. The results of a comparative test of this application in the virtual cluster environment Slurm under the control of OpenStack and in the node of cluster SKIT-4.5 in the V.M. Glushkov Institute of Cybernetics of the NAS of Ukraine are given. Information on solving the problem of finding the optimal in terms of saving resources scaling parameters for the developed application in two comparable cluster environments is given. Some features of the use of these cluster environments are clarified, in particular, a comparison of the dependence of the application speed on the number of parallel processes for two cluster environments is presented. Empirical data are presented in graphical form, which illustrate the nature of the load on the OpenStack server and the use of RAM on the number of parallel processes. Possibilities of portability between the specified cluster environments, scaling of calculations and maintenance of reproducibility of calculations for the offered test application are demonstrated. The advantages of using OpenStack technology for scientific biomedical calculations are pointed out. Conclusions. The described example of test deployment and use of OpenStack gives an idea of the requirements for the necessary technical base to ensure the reproducibility of scientific biomedical calculations in cloud and cluster environments.

Keywords

• cloud technologies
• reproducible calculations
• cluster platform