MLOps Components, Tools, Process, and Metrics: A Systematic Literature Review

Abstract

With the growing popularity of machine learning, implementations of the environment for developing and maintaining these models, called MLOps, are becoming more common. The number of publications in this area is relatively small, although growing rapidly. Our goal was to review the current state of the literature in the MLOps area and answer the following research questions: What classes of tools are used in MLOps environments? Which tool implementations are the most popular? What processes are implemented within MLOps? What metrics are used to measure the effectiveness of MLOps implementation? Based on this review, we identified classes of tools included in the MLOps architecture, along with their most popular implementations. While some tools originate from DevOps practices, others, such as Model Orchestrators, Feature Stores, and Model Repositories, are unique to MLOps. We propose a reference MLOps architecture based on these findings and outline the stages of the model production process. We also sought metrics that would allow us to assess and compare the effectiveness of MLOps practices, but unfortunately, we were unable to find a satisfactory answer in this area.

Keywords

• Architecture
• machine learning operations
• MLOps