Leveraging Large Language Models for Software Model Completion: Results from Industrial and Public Datasets

• URL: http://arxiv.org/abs/2406.17651v2
• Date: Wed, 26 Jun 2024 17:43:15 GMT
• Title: Leveraging Large Language Models for Software Model Completion: Results from Industrial and Public Datasets
• Authors: Christof Tinnes, Alisa Welter, Sven Apel,
• Abstract summary: We evaluate the potential of large language models for model completion with retrieval-augmented generation. We found that large language models are indeed a promising technology for supporting software model evolution.
• Score: 6.585732390922304
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modeling structure and behavior of software systems plays a crucial role in the industrial practice of software engineering. As with other software engineering artifacts, software models are subject to evolution. Supporting modelers in evolving software models with recommendations for model completions is still an open problem, though. In this paper, we explore the potential of large language models for this task. In particular, we propose an approach, retrieval-augmented generation, leveraging large language models, model histories, and retrieval-augmented generation for model completion. Through experiments on three datasets, including an industrial application, one public open-source community dataset, and one controlled collection of simulated model repositories, we evaluate the potential of large language models for model completion with retrieval-augmented generation. We found that large language models are indeed a promising technology for supporting software model evolution (62.30% semantically correct completions on real-world industrial data and up to 86.19% type-correct completions). The general inference capabilities of large language models are particularly useful when dealing with concepts for which there are few, noisy, or no examples at all.

Related papers

• EvolveDirector: Approaching Advanced Text-to-Image Generation with Large Vision-Language Models [36.576853882830896]
  We introduce EvolveDirector to train a text-to-image generation model comparable to advanced models using publicly available resources. This framework interacts with advanced models through their public APIs to obtain text-image data pairs to train a base model. We leverage pre-trained large vision-language models (VLMs) to guide the evolution of the base model.
  arXiv  Detail & Related papers  (2024-10-09T17:52:28Z)
• Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities [89.40778301238642]
  Model merging is an efficient empowerment technique in the machine learning community. There is a significant gap in the literature regarding a systematic and thorough review of these techniques.
  arXiv  Detail & Related papers  (2024-08-14T16:58:48Z)
• Apple Intelligence Foundation Language Models [109.60033785567484]
  This report describes the model architecture, the data used to train the model, the training process, and the evaluation results. We highlight our focus on Responsible AI and how the principles are applied throughout the model development.
  arXiv  Detail & Related papers  (2024-07-29T18:38:49Z)
• Data-Juicer Sandbox: A Comprehensive Suite for Multimodal Data-Model Co-development [67.55944651679864]
  We present a novel sandbox suite tailored for integrated data-model co-development. This sandbox provides a comprehensive experimental platform, enabling rapid iteration and insight-driven refinement of both data and models. We also uncover fruitful insights gleaned from exhaustive benchmarks, shedding light on the critical interplay between data quality, diversity, and model behavior.
  arXiv  Detail & Related papers  (2024-07-16T14:40:07Z)
• Knowledge Fusion By Evolving Weights of Language Models [5.354527640064584]
  This paper examines the approach of integrating multiple models into a unified model. We propose a knowledge fusion method named Evolver, inspired by evolutionary algorithms.
  arXiv  Detail & Related papers  (2024-06-18T02:12:34Z)
• Towards Automatic Support of Software Model Evolution with Large Language~Models [6.872484164111954]
  We propose an approach that utilizes large language models for model completion and discovering editing patterns in model histories of software systems. We have found that large language models are indeed a promising technology for supporting software model evolution.
  arXiv  Detail & Related papers  (2023-12-19T18:38:01Z)
• MindLLM: Pre-training Lightweight Large Language Model from Scratch, Evaluations and Domain Applications [46.337078949637345]
  We present MindLLM, a novel series of bilingual lightweight large language models, trained from scratch. A thorough account of experiences accrued during large model development is given, covering every step of the process. MindLLM consistently matches or surpasses the performance of other open-source larger models on some public benchmarks.
  arXiv  Detail & Related papers  (2023-10-24T12:22:34Z)
• Adapting Large Language Models for Content Moderation: Pitfalls in Data Engineering and Supervised Fine-tuning [79.53130089003986]
  Large Language Models (LLMs) have become a feasible solution for handling tasks in various domains. In this paper, we introduce how to fine-tune a LLM model that can be privately deployed for content moderation.
  arXiv  Detail & Related papers  (2023-10-05T09:09:44Z)
• Opportunities for Large Language Models and Discourse in Engineering Design [0.0]
  We argue that discourse should be regarded as the core of engineering design processes, and therefore should be represented in a digital artifact. We describe how simulations, experiments, topology optimizations, and other process steps can be integrated into a machine-actionable, discourse-centric design process.
  arXiv  Detail & Related papers  (2023-06-15T14:46:44Z)
• Language Models are General-Purpose Interfaces [109.45478241369655]
  We propose to use language models as a general-purpose interface to various foundation models. A collection of pretrained encoders perceive diverse modalities (such as vision, and language) We propose a semi-causal language modeling objective to jointly pretrain the interface and the modular encoders.
  arXiv  Detail & Related papers  (2022-06-13T17:34:22Z)
• Model Reprogramming: Resource-Efficient Cross-Domain Machine Learning [65.268245109828]
  In data-rich domains such as vision, language, and speech, deep learning prevails to deliver high-performance task-specific models. Deep learning in resource-limited domains still faces multiple challenges including (i) limited data, (ii) constrained model development cost, and (iii) lack of adequate pre-trained models for effective finetuning. Model reprogramming enables resource-efficient cross-domain machine learning by repurposing a well-developed pre-trained model from a source domain to solve tasks in a target domain without model finetuning.
  arXiv  Detail & Related papers  (2022-02-22T02:33:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.