Quality Assurance in MLOps Setting: An Industrial Perspective
- URL: http://arxiv.org/abs/2211.12706v2
- Date: Thu, 24 Nov 2022 19:29:46 GMT
- Title: Quality Assurance in MLOps Setting: An Industrial Perspective
- Authors: Ayan Chatterjee, Bestoun S. Ahmed, Erik Hallin, Anton Engman
- Abstract summary: Machine learning (ML) is widely used in industry to provide the core functionality of production systems.
Due to production demand and time constraints, automated software engineering practices are highly applicable.
This paper examines the QA challenges that arise in industrial MLOps and conceptualizes modular strategies to deal with data integrity and Data Quality.
- Score: 0.11470070927586014
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Today, machine learning (ML) is widely used in industry to provide the core
functionality of production systems. However, it is practically always used in
production systems as part of a larger end-to-end software system that is made
up of several other components in addition to the ML model. Due to production
demand and time constraints, automated software engineering practices are
highly applicable. The increased use of automated ML software engineering
practices in industries such as manufacturing and utilities requires an
automated Quality Assurance (QA) approach as an integral part of ML software.
Here, QA helps reduce risk by offering an objective perspective on the software
task. Although conventional software engineering has automated tools for QA
data analysis for data-driven ML, the use of QA practices for ML in operation
(MLOps) is lacking. This paper examines the QA challenges that arise in
industrial MLOps and conceptualizes modular strategies to deal with data
integrity and Data Quality (DQ). The paper is accompanied by real industrial
use-cases from industrial partners. The paper also presents several challenges
that may serve as a basis for future studies.
Related papers
- A Cyber Manufacturing IoT System for Adaptive Machine Learning Model Deployment by Interactive Causality Enabled Self-Labeling [0.0]
This paper proposes the AdaptIoT system, comprised of an end-to-end data streaming pipeline, ML service integration, and an automated self-labeling service.
The self-labeling service consists of causal knowledge bases and automated full-cycle self-labeling to adapt multiple ML models simultaneously.
A field demonstration of a self-labeling adaptive ML application is conducted with a makerspace and shows reliable performance.
arXiv Detail & Related papers (2024-04-09T03:10:45Z) - Quantum Computing Enhanced Service Ecosystem for Simulation in Manufacturing [56.61654656648898]
We propose a framework for a quantum computing-enhanced service ecosystem for simulation in manufacturing.
We analyse two high-value use cases with the aim of a quantitative evaluation of these new computing paradigms for industrially-relevant settings.
arXiv Detail & Related papers (2024-01-19T11:04:14Z) - Machine Learning Meets Advanced Robotic Manipulation [48.6221343014126]
The paper reviews cutting edge technologies and recent trends on machine learning methods applied to real-world manipulation tasks.
The rest of the paper is devoted to ML applications in different domains such as industry, healthcare, agriculture, space, military, and search and rescue.
arXiv Detail & Related papers (2023-09-22T01:06:32Z) - Automated Machine Learning in the smart construction era:Significance
and accessibility for industrial classification and regression tasks [6.206133097433925]
This paper explores the application of automated machine learning (AutoML) techniques to the construction industry.
By leveraging AutoML, construction professionals can now utilize software to process industrial data into ML models that assist in project management.
arXiv Detail & Related papers (2023-08-03T03:17:22Z) - Benchmarking Automated Machine Learning Methods for Price Forecasting
Applications [58.720142291102135]
We show the possibility of substituting manually created ML pipelines with automated machine learning (AutoML) solutions.
Based on the CRISP-DM process, we split the manual ML pipeline into a machine learning and non-machine learning part.
We show in a case study for the industrial use case of price forecasting, that domain knowledge combined with AutoML can weaken the dependence on ML experts.
arXiv Detail & Related papers (2023-04-28T10:27:38Z) - OmniForce: On Human-Centered, Large Model Empowered and Cloud-Edge
Collaborative AutoML System [85.8338446357469]
We introduce OmniForce, a human-centered AutoML system that yields both human-assisted ML and ML-assisted human techniques.
We show how OmniForce can put an AutoML system into practice and build adaptive AI in open-environment scenarios.
arXiv Detail & Related papers (2023-03-01T13:35:22Z) - Machine Learning for Software Engineering: A Tertiary Study [13.832268599253412]
Machine learning (ML) techniques increase the effectiveness of software engineering (SE) lifecycle activities.
We systematically collected, quality-assessed, summarized, and categorized 83 reviews in ML for SE published between 2009-2022, covering 6,117 primary studies.
The SE areas most tackled with ML are software quality and testing, while human-centered areas appear more challenging for ML.
arXiv Detail & Related papers (2022-11-17T09:19:53Z) - Enabling Automated Machine Learning for Model-Driven AI Engineering [60.09869520679979]
We propose a novel approach to enable Model-Driven Software Engineering and Model-Driven AI Engineering.
In particular, we support Automated ML, thus assisting software engineers without deep AI knowledge in developing AI-intensive systems.
arXiv Detail & Related papers (2022-03-06T10:12:56Z) - Quality Management of Machine Learning Systems [0.0]
Artificial Intelligence (AI) has become a part of our daily lives due to major advances in Machine Learning (ML) techniques.
For business/mission-critical systems, serious concerns about reliability and maintainability of AI applications remain.
This paper presents a view of a holistic quality management framework for ML applications based on the current advances.
arXiv Detail & Related papers (2020-06-16T21:34:44Z) - Towards CRISP-ML(Q): A Machine Learning Process Model with Quality
Assurance Methodology [53.063411515511056]
We propose a process model for the development of machine learning applications.
The first phase combines business and data understanding as data availability oftentimes affects the feasibility of the project.
The sixth phase covers state-of-the-art approaches for monitoring and maintenance of a machine learning applications.
arXiv Detail & Related papers (2020-03-11T08:25:49Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.