Engineering AI Systems: A Research Agenda

• URL: http://arxiv.org/abs/2001.07522v2
• Date: Wed, 3 Jun 2020 12:59:36 GMT
• Title: Engineering AI Systems: A Research Agenda
• Authors: Jan Bosch, Ivica Crnkovic, Helena Holmstr\"om Olsson
• Abstract summary: We provide a conceptualization of the typical evolution patterns that companies experience when employing machine learning. The main contribution of the paper is a research agenda for AI engineering that provides an overview of the key engineering challenges surrounding ML solutions.
• Score: 9.84673609667263
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Artificial intelligence (AI) and machine learning (ML) are increasingly broadly adopted in industry, However, based on well over a dozen case studies, we have learned that deploying industry-strength, production quality ML models in systems proves to be challenging. Companies experience challenges related to data quality, design methods and processes, performance of models as well as deployment and compliance. We learned that a new, structured engineering approach is required to construct and evolve systems that contain ML/DL components. In this paper, we provide a conceptualization of the typical evolution patterns that companies experience when employing ML as well as an overview of the key problems experienced by the companies that we have studied. The main contribution of the paper is a research agenda for AI engineering that provides an overview of the key engineering challenges surrounding ML solutions and an overview of open items that need to be addressed by the research community at large.

Related papers

• Search, Verify and Feedback: Towards Next Generation Post-training Paradigm of Foundation Models via Verifier Engineering [51.31836988300326]
  Verifier engineering is a novel post-training paradigm specifically designed for the era of foundation models. We systematically categorize the verifier engineering process into three essential stages: search, verify, and feedback.
  arXiv  Detail & Related papers  (2024-11-18T12:04:52Z)
• Towards Trustworthy Machine Learning in Production: An Overview of the Robustness in MLOps Approach [0.0]
  In recent years, AI researchers and practitioners have introduced principles and guidelines to build systems that make reliable and trustworthy decisions. In practice, a fundamental challenge arises when the system needs to be operationalized and deployed to evolve and operate in real-life environments continuously. To address this challenge, Machine Learning Operations (MLOps) have emerged as a potential recipe for standardizing ML solutions in deployment.
  arXiv  Detail & Related papers  (2024-10-28T09:34:08Z)
• Naming the Pain in Machine Learning-Enabled Systems Engineering [8.092979562919878]
  Machine learning (ML)-enabled systems are being increasingly adopted by companies. This paper aims to deliver a comprehensive overview of the current status quo of engineering ML-enabled systems.
  arXiv  Detail & Related papers  (2024-05-20T06:59:20Z)
• Enabling Automated Machine Learning for Model-Driven AI Engineering [60.09869520679979]
  We propose a novel approach to enable Model-Driven Software Engineering and Model-Driven AI Engineering. In particular, we support Automated ML, thus assisting software engineers without deep AI knowledge in developing AI-intensive systems.
  arXiv  Detail & Related papers  (2022-03-06T10:12:56Z)
• Panoramic Learning with A Standardized Machine Learning Formalism [116.34627789412102]
  This paper presents a standardized equation of the learning objective, that offers a unifying understanding of diverse ML algorithms. It also provides guidance for mechanic design of new ML solutions, and serves as a promising vehicle towards panoramic learning with all experiences.
  arXiv  Detail & Related papers  (2021-08-17T17:44:38Z)
• Engineering an Intelligent Essay Scoring and Feedback System: An Experience Report [1.5168188294440734]
  We describe an exploratory system for assessing the quality of essays supplied by customers of a specialized recruitment support service. The problem domain is challenging because the open-ended customer-supplied source text has considerable scope for ambiguity and error. There is also a need to incorporate specialized business domain knowledge into the intelligent processing systems.
  arXiv  Detail & Related papers  (2021-03-25T03:46:05Z)
• Technology Readiness Levels for Machine Learning Systems [107.56979560568232]
  Development and deployment of machine learning systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end. We have developed a proven systems engineering approach for machine learning development and deployment. Our "Machine Learning Technology Readiness Levels" framework defines a principled process to ensure robust, reliable, and responsible systems.
  arXiv  Detail & Related papers  (2021-01-11T15:54:48Z)
• A Software Engineering Perspective on Engineering Machine Learning Systems: State of the Art and Challenges [0.0]
  Advancements in machine learning (ML) lead to a shift from the traditional view of software development, where algorithms are hard-coded by humans, to ML systems materialized through learning from data. We need to revisit our ways of developing software systems and consider the particularities required by these new types of systems.
  arXiv  Detail & Related papers  (2020-12-14T20:06:31Z)
• Software engineering for artificial intelligence and machine learning software: A systematic literature review [6.681725960709127]
  This study aims to investigate how software engineering has been applied in the development of AI/ML systems. Main challenges faced by professionals are in areas of testing, AI software quality, and data management.
  arXiv  Detail & Related papers  (2020-11-07T11:06:28Z)
• Technology Readiness Levels for AI & ML [79.22051549519989]
  Development of machine learning systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end. Engineering systems follow well-defined processes and testing standards to streamline development for high-quality, reliable results. We propose a proven systems engineering approach for machine learning development and deployment.
  arXiv  Detail & Related papers  (2020-06-21T17:14:34Z)
• Towards CRISP-ML(Q): A Machine Learning Process Model with Quality Assurance Methodology [53.063411515511056]
  We propose a process model for the development of machine learning applications. The first phase combines business and data understanding as data availability oftentimes affects the feasibility of the project. The sixth phase covers state-of-the-art approaches for monitoring and maintenance of a machine learning applications.
  arXiv  Detail & Related papers  (2020-03-11T08:25:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.