On Augmenting Scenario-Based Modeling with Generative AI

• URL: http://arxiv.org/abs/2401.02245v1
• Date: Thu, 4 Jan 2024 12:58:25 GMT
• Title: On Augmenting Scenario-Based Modeling with Generative AI
• Authors: David Harel, Guy Katz, Assaf Marron, Smadar Szekely
• Abstract summary: We outline a method for the safer and more structured use of chatbots as part of the modeling process. We propose leveraging scenario-based modeling techniques, which are known to facilitate the automated analysis of models. We describe favorable preliminary results, which highlight the potential of this approach.
• Score: 1.4501446815590895
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The manual modeling of complex systems is a daunting task; and although a plethora of methods exist that mitigate this issue, the problem remains very difficult. Recent advances in generative AI have allowed the creation of general-purpose chatbots, capable of assisting software engineers in various modeling tasks. However, these chatbots are often inaccurate, and an unstructured use thereof could result in erroneous system models. In this paper, we outline a method for the safer and more structured use of chatbots as part of the modeling process. To streamline this integration, we propose leveraging scenario-based modeling techniques, which are known to facilitate the automated analysis of models. We argue that through iterative invocations of the chatbot and the manual and automatic inspection of the resulting models, a more accurate system model can eventually be obtained. We describe favorable preliminary results, which highlight the potential of this approach.

Related papers

• Towards Learning Stochastic Population Models by Gradient Descent [0.0]
  We show that simultaneous estimation of parameters and structure poses major challenges for optimization procedures. We demonstrate accurate estimation of models but find that enforcing the inference of parsimonious, interpretable models drastically increases the difficulty.
  arXiv  Detail & Related papers  (2024-04-10T14:38:58Z)
• Automated Process Planning Based on a Semantic Capability Model and SMT [50.76251195257306]
  In research of manufacturing systems and autonomous robots, the term capability is used for a machine-interpretable specification of a system function. We present an approach that combines these two topics: starting from a semantic capability model, an AI planning problem is automatically generated.
  arXiv  Detail & Related papers  (2023-12-14T10:37:34Z)
• QualEval: Qualitative Evaluation for Model Improvement [82.73561470966658]
  We propose QualEval, which augments quantitative scalar metrics with automated qualitative evaluation as a vehicle for model improvement. QualEval uses a powerful LLM reasoner and our novel flexible linear programming solver to generate human-readable insights. We demonstrate that leveraging its insights, for example, improves the absolute performance of the Llama 2 model by up to 15% points relative.
  arXiv  Detail & Related papers  (2023-11-06T00:21:44Z)
• Fine-Tuning Generative Models as an Inference Method for Robotic Tasks [18.745665662647912]
  We investigate how to quickly adapt the sample generation of neural network models to observations in robotic tasks. The key idea is to quickly fine-tune the model by fitting it to generated samples matching the observed evidence. We show that our method can be applied to both autoregressive models and variational autoencoders.
  arXiv  Detail & Related papers  (2023-10-19T16:11:49Z)
• Representing Timed Automata and Timing Anomalies of Cyber-Physical Production Systems in Knowledge Graphs [51.98400002538092]
  This paper aims to improve model-based anomaly detection in CPPS by combining the learned timed automaton with a formal knowledge graph about the system. Both the model and the detected anomalies are described in the knowledge graph in order to allow operators an easier interpretation of the model and the detected anomalies.
  arXiv  Detail & Related papers  (2023-08-25T15:25:57Z)
• Conversational Process Modeling: Can Generative AI Empower Domain Experts in Creating and Redesigning Process Models? [0.0]
  This work provides a systematic analysis of existing chatbots for support of conversational process modeling. A literature review on conversational process modeling is performed, resulting in a taxonomy of application scenarios for conversational process modeling. An evaluation method is applied for the output of AI-driven chatbots with respect to completeness and correctness of the process models.
  arXiv  Detail & Related papers  (2023-04-19T06:54:14Z)
• Real-to-Sim: Predicting Residual Errors of Robotic Systems with Sparse Data using a Learning-based Unscented Kalman Filter [65.93205328894608]
  We learn the residual errors between a dynamic and/or simulator model and the real robot. We show that with the learned residual errors, we can further close the reality gap between dynamic models, simulations, and actual hardware.
  arXiv  Detail & Related papers  (2022-09-07T15:15:12Z)
• Few-shot Prompting Towards Controllable Response Generation [49.479958672988566]
  We first explored the combination of prompting and reinforcement learning (RL) to steer models' generation without accessing any of the models' parameters. We apply multi-task learning to make the model learn to generalize to new tasks better. Experiment results show that our proposed method can successfully control several state-of-the-art (SOTA) dialogue models without accessing their parameters.
  arXiv  Detail & Related papers  (2022-06-08T14:48:06Z)
• Model Reprogramming: Resource-Efficient Cross-Domain Machine Learning [65.268245109828]
  In data-rich domains such as vision, language, and speech, deep learning prevails to deliver high-performance task-specific models. Deep learning in resource-limited domains still faces multiple challenges including (i) limited data, (ii) constrained model development cost, and (iii) lack of adequate pre-trained models for effective finetuning. Model reprogramming enables resource-efficient cross-domain machine learning by repurposing a well-developed pre-trained model from a source domain to solve tasks in a target domain without model finetuning.
  arXiv  Detail & Related papers  (2022-02-22T02:33:54Z)
• High-Accuracy Model-Based Reinforcement Learning, a Survey [2.0196229393131726]
  Deep reinforcement learning has shown remarkable success in game playing and robotics. To reduce the number of environment samples, model-based reinforcement learning creates an explicit model of the environment dynamics. Some of these methods succeed in achieving high accuracy at low sample complexity, most do so either in a robotics or in a games context.
  arXiv  Detail & Related papers  (2021-07-17T14:01:05Z)
• Structured Mechanical Models for Robot Learning and Control [38.52004843488286]
  Black-box neural networks suffer from data-inefficiency and the difficulty to incorporate prior knowledge. We introduce Structured Mechanical Models that are data-efficient, easily amenable to prior knowledge, and easily usable with model-based control techniques. We demonstrate that they generalize better from limited data and yield more reliable model-based controllers on a variety of simulated robotic domains.
  arXiv  Detail & Related papers  (2020-04-21T21:12:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.