Technology Readiness Levels for Machine Learning Systems

• URL: http://arxiv.org/abs/2101.03989v1
• Date: Mon, 11 Jan 2021 15:54:48 GMT
• Title: Technology Readiness Levels for Machine Learning Systems
• Authors: Alexander Lavin, Ciar\'an M. Gilligan-Lee, Alessya Visnjic, Siddha Ganju, Dava Newman, Sujoy Ganguly, Danny Lange, At{\i}l{\i}m G\"une\c{s} Baydin, Amit Sharma, Adam Gibson, Yarin Gal, Eric P. Xing, Chris Mattmann, James Parr
• Abstract summary: Development and deployment of machine learning systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end. We have developed a proven systems engineering approach for machine learning development and deployment. Our "Machine Learning Technology Readiness Levels" framework defines a principled process to ensure robust, reliable, and responsible systems.
• Score: 107.56979560568232
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The development and deployment of machine learning (ML) systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end. The lack of diligence can lead to technical debt, scope creep and misaligned objectives, model misuse and failures, and expensive consequences. Engineering systems, on the other hand, follow well-defined processes and testing standards to streamline development for high-quality, reliable results. The extreme is spacecraft systems, where mission critical measures and robustness are ingrained in the development process. Drawing on experience in both spacecraft engineering and ML (from research through product across domain areas), we have developed a proven systems engineering approach for machine learning development and deployment. Our "Machine Learning Technology Readiness Levels" (MLTRL) framework defines a principled process to ensure robust, reliable, and responsible systems while being streamlined for ML workflows, including key distinctions from traditional software engineering. Even more, MLTRL defines a lingua franca for people across teams and organizations to work collaboratively on artificial intelligence and machine learning technologies. Here we describe the framework and elucidate it with several real world use-cases of developing ML methods from basic research through productization and deployment, in areas such as medical diagnostics, consumer computer vision, satellite imagery, and particle physics.

Related papers

• Towards Trustworthy Machine Learning in Production: An Overview of the Robustness in MLOps Approach [0.0]
  In recent years, AI researchers and practitioners have introduced principles and guidelines to build systems that make reliable and trustworthy decisions. In practice, a fundamental challenge arises when the system needs to be operationalized and deployed to evolve and operate in real-life environments continuously. To address this challenge, Machine Learning Operations (MLOps) have emerged as a potential recipe for standardizing ML solutions in deployment.
  arXiv  Detail & Related papers  (2024-10-28T09:34:08Z)
• A Framework to Model ML Engineering Processes [1.9744907811058787]
  Development of Machine Learning (ML) based systems is complex and requires multidisciplinary teams with diverse skill sets. Current process modeling languages are not suitable for describing the development of such systems. We introduce a framework for modeling ML-based software development processes, built around a domain-specific language.
  arXiv  Detail & Related papers  (2024-04-29T09:17:36Z)
• Machine Learning Meets Advanced Robotic Manipulation [48.6221343014126]
  The paper reviews cutting edge technologies and recent trends on machine learning methods applied to real-world manipulation tasks. The rest of the paper is devoted to ML applications in different domains such as industry, healthcare, agriculture, space, military, and search and rescue.
  arXiv  Detail & Related papers  (2023-09-22T01:06:32Z)
• Achieving Guidance in Applied Machine Learning through Software Engineering Techniques [0.0]
  We look at the (lack of) guidance that currently used development environments and ML APIs provide to developers of ML applications. We show that current ML tools fall short of fulfilling some basic software engineering gold standards. Our findings point out ample opportunities for research on ML-specific software engineering.
  arXiv  Detail & Related papers  (2022-03-29T12:54:57Z)
• Panoramic Learning with A Standardized Machine Learning Formalism [116.34627789412102]
  This paper presents a standardized equation of the learning objective, that offers a unifying understanding of diverse ML algorithms. It also provides guidance for mechanic design of new ML solutions, and serves as a promising vehicle towards panoramic learning with all experiences.
  arXiv  Detail & Related papers  (2021-08-17T17:44:38Z)
• Towards a Robust and Trustworthy Machine Learning System Development [0.09236074230806578]
  We present our recent survey on the state-of-the-art ML trustworthiness and technologies from a security engineering perspective. We then push our studies forward above and beyond a survey by describing a metamodel we created that represents the body of knowledge in a standard and visualized way for ML practitioners. We propose future research directions motivated by our findings to advance the development of robust and trustworthy ML systems.
  arXiv  Detail & Related papers  (2021-01-08T14:43:58Z)
• Technology Readiness Levels for AI & ML [79.22051549519989]
  Development of machine learning systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end. Engineering systems follow well-defined processes and testing standards to streamline development for high-quality, reliable results. We propose a proven systems engineering approach for machine learning development and deployment.
  arXiv  Detail & Related papers  (2020-06-21T17:14:34Z)
• Machine Learning for Software Engineering: A Systematic Mapping [73.30245214374027]
  The software development industry is rapidly adopting machine learning for transitioning modern day software systems towards highly intelligent and self-learning systems. No comprehensive study exists that explores the current state-of-the-art on the adoption of machine learning across software engineering life cycle stages. This study introduces a machine learning for software engineering (MLSE) taxonomy classifying the state-of-the-art machine learning techniques according to their applicability to various software engineering life cycle stages.
  arXiv  Detail & Related papers  (2020-05-27T11:56:56Z)
• Towards CRISP-ML(Q): A Machine Learning Process Model with Quality Assurance Methodology [53.063411515511056]
  We propose a process model for the development of machine learning applications. The first phase combines business and data understanding as data availability oftentimes affects the feasibility of the project. The sixth phase covers state-of-the-art approaches for monitoring and maintenance of a machine learning applications.
  arXiv  Detail & Related papers  (2020-03-11T08:25:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.