Towards CRISP-ML(Q): A Machine Learning Process Model with Quality Assurance Methodology

• URL: http://arxiv.org/abs/2003.05155v2
• Date: Wed, 24 Feb 2021 14:33:24 GMT
• Title: Towards CRISP-ML(Q): A Machine Learning Process Model with Quality Assurance Methodology
• Authors: Stefan Studer, Thanh Binh Bui, Christian Drescher, Alexander Hanuschkin, Ludwig Winkler, Steven Peters, Klaus-Robert Mueller
• Abstract summary: We propose a process model for the development of machine learning applications. The first phase combines business and data understanding as data availability oftentimes affects the feasibility of the project. The sixth phase covers state-of-the-art approaches for monitoring and maintenance of a machine learning applications.
• Score: 53.063411515511056
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Machine learning is an established and frequently used technique in industry and academia but a standard process model to improve success and efficiency of machine learning applications is still missing. Project organizations and machine learning practitioners have a need for guidance throughout the life cycle of a machine learning application to meet business expectations. We therefore propose a process model for the development of machine learning applications, that covers six phases from defining the scope to maintaining the deployed machine learning application. The first phase combines business and data understanding as data availability oftentimes affects the feasibility of the project. The sixth phase covers state-of-the-art approaches for monitoring and maintenance of a machine learning applications, as the risk of model degradation in a changing environment is eminent. With each task of the process, we propose quality assurance methodology that is suitable to adress challenges in machine learning development that we identify in form of risks. The methodology is drawn from practical experience and scientific literature and has proven to be general and stable. The process model expands on CRISP-DM, a data mining process model that enjoys strong industry support but lacks to address machine learning specific tasks. Our work proposes an industry and application neutral process model tailored for machine learning applications with focus on technical tasks for quality assurance.

Related papers

• Application-Driven Innovation in Machine Learning [56.85396167616353]
  We describe the paradigm of application-driven research in machine learning. We show how this approach can productively synergize with methods-driven work. Despite these benefits, we find that reviewing, hiring, and teaching practices in machine learning often hold back application-driven innovation.
  arXiv  Detail & Related papers  (2024-03-26T04:59:27Z)
• Machine Learning Meets Advanced Robotic Manipulation [48.6221343014126]
  The paper reviews cutting edge technologies and recent trends on machine learning methods applied to real-world manipulation tasks. The rest of the paper is devoted to ML applications in different domains such as industry, healthcare, agriculture, space, military, and search and rescue.
  arXiv  Detail & Related papers  (2023-09-22T01:06:32Z)
• Concepts for Automated Machine Learning in Smart Grid Applications [0.2624902795082451]
  Large-scale application of machine learning methods in energy systems is impaired by the need for expert knowledge. Process knowledge is required for the problem formalization, as well as the model validation and application. We define five levels of automation for forecasting in alignment with the SAE standard for autonomous vehicles.
  arXiv  Detail & Related papers  (2021-10-26T11:34:41Z)
• Multi Agent System for Machine Learning Under Uncertainty in Cyber Physical Manufacturing System [78.60415450507706]
  Recent advancements in predictive machine learning has led to its application in various use cases in manufacturing. Most research focused on maximising predictive accuracy without addressing the uncertainty associated with it. In this paper, we determine the sources of uncertainty in machine learning and establish the success criteria of a machine learning system to function well under uncertainty.
  arXiv  Detail & Related papers  (2021-07-28T10:28:05Z)
• An Experience Report on Machine Learning Reproducibility: Guidance for Practitioners and TensorFlow Model Garden Contributors [1.177923904173852]
  This report is to define a process for reproducing a state-of-the-art machine learning model at a level of quality suitable for inclusion in the Model Garden. We report on our experiences implementing the YOLO model family with a team of 26 student researchers, share the tools we developed, and describe the lessons we learned along the way.
  arXiv  Detail & Related papers  (2021-07-02T04:32:18Z)
• Machine Learning Model Development from a Software Engineering Perspective: A Systematic Literature Review [0.0]
  Data scientists often develop machine learning models to solve a variety of problems in the industry and academy. This paper is an effort to investigate the challenges and practices that emerge during the development of ML models from the software engineering perspective.
  arXiv  Detail & Related papers  (2021-02-15T14:25:13Z)
• Technology Readiness Levels for Machine Learning Systems [107.56979560568232]
  Development and deployment of machine learning systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end. We have developed a proven systems engineering approach for machine learning development and deployment. Our "Machine Learning Technology Readiness Levels" framework defines a principled process to ensure robust, reliable, and responsible systems.
  arXiv  Detail & Related papers  (2021-01-11T15:54:48Z)
• Technology Readiness Levels for AI & ML [79.22051549519989]
  Development of machine learning systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end. Engineering systems follow well-defined processes and testing standards to streamline development for high-quality, reliable results. We propose a proven systems engineering approach for machine learning development and deployment.
  arXiv  Detail & Related papers  (2020-06-21T17:14:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.