Toward Methodical Discovery and Handling of Hidden Assumptions in Complex Systems and Models

• URL: http://arxiv.org/abs/2312.16507v1
• Date: Wed, 27 Dec 2023 10:33:12 GMT
• Title: Toward Methodical Discovery and Handling of Hidden Assumptions in Complex Systems and Models
• Authors: David Harel, Uwe A{\ss}mann, Fabiana Fournier, Lior Limonad, Assaf Marron and Smadar Szekely
• Abstract summary: external reviews can uncover undocumented built-in assumptions. We show that a variety of digital artifacts can be automatically checked against extensive reference knowledge. We believe that systematic handling of this aspect of system engineering can contribute significantly to the quality and safety of complex systems and models.
• Score: 3.1771791275364194
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Methodologies for development of complex systems and models include external reviews by domain and technology experts. Among others, such reviews can uncover undocumented built-in assumptions that may be critical for correct and safe operation or constrain applicability. Since such assumptions may still escape human-centered processes like reviews, agile development, and risk analyses, here, we contribute toward making this process more methodical and automatable. We first present a blueprint for a taxonomy and formalization of the problem. We then show that a variety of digital artifacts of the system or model can be automatically checked against extensive reference knowledge. Since mimicking the breadth and depth of knowledge and skills of experts may appear unattainable, we illustrate the basic feasibility of automation with rudimentary experiments using OpenAI's ChatGPT. We believe that systematic handling of this aspect of system engineering can contribute significantly to the quality and safety of complex systems and models, and to the efficiency of development projects. We dedicate this work to Werner Damm, whose contributions to modeling and model-based development, in industry and academia, with a special focus on safety, helped establish a solid foundation to our discipline and to the work of many scientists and professionals, including, naturally, the approaches and techniques described here.

Related papers

• Data Analysis in the Era of Generative AI [56.44807642944589]
  This paper explores the potential of AI-powered tools to reshape data analysis, focusing on design considerations and challenges. We explore how the emergence of large language and multimodal models offers new opportunities to enhance various stages of data analysis workflow. We then examine human-centered design principles that facilitate intuitive interactions, build user trust, and streamline the AI-assisted analysis workflow across multiple apps.
  arXiv  Detail & Related papers  (2024-09-27T06:31:03Z)
• The Contribution of XAI for the Safe Development and Certification of AI: An Expert-Based Analysis [4.119574613934122]
  The black-box nature of machine learning models limits the use of conventional avenues of approach towards certifying complex technical systems. As a potential solution, methods to give insights into this black-box could be used. We find that XAI methods can be a helpful asset for safe AI development, but since certification relies on comprehensive and correct information about technical systems, their impact is expected to be limited.
  arXiv  Detail & Related papers  (2024-07-22T16:08:21Z)
• Implementing a hybrid approach in a knowledge engineering process to manage technical advice relating to feedback from the operation of complex sensitive equipment [0.0]
  This article explains how an industrial company in the nuclear and defense sectors adopted such an approach. It builds a complete system with a "SARBACANES" application to support its business processes and perpetuate its know-how and expertise in a knowledge base.
  arXiv  Detail & Related papers  (2024-07-08T08:17:10Z)
• Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review [1.6006550105523192]
  Review explores the pivotal role of prompt engineering in unleashing the capabilities of Large Language Models (LLMs) Examines both foundational and advanced methodologies of prompt engineering, including techniques such as self-consistency, chain-of-thought, and generated knowledge. Review also reflects the essential role of prompt engineering in advancing AI capabilities, providing a structured framework for future research and application.
  arXiv  Detail & Related papers  (2023-10-23T09:15:18Z)
• Constrained Reinforcement Learning for Robotics via Scenario-Based Programming [64.07167316957533]
  It is crucial to optimize the performance of DRL-based agents while providing guarantees about their behavior. This paper presents a novel technique for incorporating domain-expert knowledge into a constrained DRL training loop. Our experiments demonstrate that using our approach to leverage expert knowledge dramatically improves the safety and the performance of the agent.
  arXiv  Detail & Related papers  (2022-06-20T07:19:38Z)
• Scenario-Assisted Deep Reinforcement Learning [3.5036351567024275]
  We propose a technique for enhancing the reinforcement learning training process. It allows human engineers to directly contribute their expert knowledge, making the agent under training more likely to comply with relevant constraints. We evaluate our technique using a case-study from the domain of internet congestion control.
  arXiv  Detail & Related papers  (2022-02-09T08:46:13Z)
• Multi Agent System for Machine Learning Under Uncertainty in Cyber Physical Manufacturing System [78.60415450507706]
  Recent advancements in predictive machine learning has led to its application in various use cases in manufacturing. Most research focused on maximising predictive accuracy without addressing the uncertainty associated with it. In this paper, we determine the sources of uncertainty in machine learning and establish the success criteria of a machine learning system to function well under uncertainty.
  arXiv  Detail & Related papers  (2021-07-28T10:28:05Z)
• Technology Readiness Levels for Machine Learning Systems [107.56979560568232]
  Development and deployment of machine learning systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end. We have developed a proven systems engineering approach for machine learning development and deployment. Our "Machine Learning Technology Readiness Levels" framework defines a principled process to ensure robust, reliable, and responsible systems.
  arXiv  Detail & Related papers  (2021-01-11T15:54:48Z)
• Knowledge as Invariance -- History and Perspectives of Knowledge-augmented Machine Learning [69.99522650448213]
  Research in machine learning is at a turning point. Research interests are shifting away from increasing the performance of highly parameterized models to exceedingly specific tasks. This white paper provides an introduction and discussion of this emerging field in machine learning research.
  arXiv  Detail & Related papers  (2020-12-21T15:07:19Z)
• Dos and Don'ts of Machine Learning in Computer Security [74.1816306998445]
  Despite great potential, machine learning in security is prone to subtle pitfalls that undermine its performance. We identify common pitfalls in the design, implementation, and evaluation of learning-based security systems. We propose actionable recommendations to support researchers in avoiding or mitigating the pitfalls where possible.
  arXiv  Detail & Related papers  (2020-10-19T13:09:31Z)
• Synergizing Domain Expertise with Self-Awareness in Software Systems: A Patternized Architecture Guideline [11.155059219430207]
  This paper highlights the importance of synergizing domain expertise and the self-awareness to enable better self-adaptation in software systems. We present a holistic framework of notions, enriched patterns and methodology, dubbed DBASES, that offers a principled guideline for the engineers.
  arXiv  Detail & Related papers  (2020-01-20T12:17:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.