Approaching automation of multiple instance orchestration of the menoci web portal

Abstract

Introduction: The menoci web portal addresses the needs of FAIR representation of biomedical basic research data and has been successfully implemented for several large consortia at Göttingen Campus. The operation of multiple menoci instances requires efficient measures to reduce administrative resource efforts. This manuscript describes our approach to automatize server operation and software updating procedures. Methods: The menoci instances are hosted on virtual machines (VM) using IT infrastructure of the local academic IT-service provider. Source code and process documentation is hosted in the Göttingen Campus GitLab service. Continuous Integration/Continuous Delivery (CI/CD) pipelines were developed to routinely build updated Docker images from latest source code revisions and the upstream Drupal Docker image. GitLab functionality for code reviews is employed, using protected branches and the “approval” feature for merge requests. Results: At the beginning, menoci development was mainly driven by the implementation of additional modules, features and optimization of user experience to fulfill the researchers’ requirements. Since the roll out of menoci to an increasing list of research consortia, we additionally focused on improving performance, software quality and enhanced automation processes. Our developed automation pipelines include updates for web server and database components, as well as the Drupal content management system and other components that together form the menoci platform. Furthermore, all menoci code enhancements are automatically distributed to all instances. Success or failing of update processes is monitored systematically to facilitate error handling. All processes are extensively documented to easily integrate new team members into administrative tasks. Discussion: Our experience indicated that automation processes are key to reduce resource efforts for technical administrative tasks. However, a high degree of automation and dependencies invoke the potential of small errors possibly leading to large effects. Therefore, tight quality control by testing and monitoring processes is necessary.

Keywords

• research data platform
• data management
• automated processes
• Continuous Integration/Continuous Delivery (CI/CD)