Training LLMs for Generating IEC 61131-3 Structured Text with Online Feedback

• URL: http://arxiv.org/abs/2410.22159v2
• Date: Wed, 30 Oct 2024 06:11:54 GMT
• Title: Training LLMs for Generating IEC 61131-3 Structured Text with Online Feedback
• Authors: Aaron Haag, Bertram Fuchs, Altay Kacan, Oliver Lohse,
• Abstract summary: This paper proposes a novel approach to training large language models (LLMs) that emphasizes improving the quality of learning data. The framework proves highly suitable for industrial automation applications and outperforms state-of-the-art models.
• Score: 0.0
• License:
• Abstract: The advent of large language models (LLMs), such as GPT-4, has enabled significant advancements in generating code across various domains. However, these models face unique challenges when generating IEC 61131-3 Structured Text (ST) code due to limited data in public training datasets and the complexity of ST language syntax. This paper proposes a novel approach to training LLMs that emphasizes improving the quality of learning data through an online process involving compiler feedback and evaluation from a secondary LLM. In this framework, the primary LLM generates new training samples, which are subsequently evaluated by a compiler for syntactical correctness and by a specialized LLM that excels at assessing semantic accuracy, though it is not optimized for code generation itself. Through iterative refinement of the training data, this approach results in marked improvements for the trained LLM, leading to higher compilation success rates and better semantic precision. As a result, the framework proves highly suitable for industrial automation applications and outperforms state-of-the-art models.

Related papers

• Genetic Instruct: Scaling up Synthetic Generation of Coding Instructions for Large Language Models [54.51932175059004]
  We introduce a scalable method for generating synthetic instructions to enhance the code generation capability of Large Language Models. The proposed algorithm, Genetic-Instruct, mimics evolutionary processes, utilizing self-instruction to create numerous synthetic samples from a limited number of seeds.
  arXiv  Detail & Related papers  (2024-07-29T20:42:59Z)
• Source Code Summarization in the Era of Large Language Models [23.715005053430957]
  Large language models (LLMs) have led to a great boost in the performance of code-related tasks. In this paper, we undertake a systematic and comprehensive study on code summarization in the era of LLMs.
  arXiv  Detail & Related papers  (2024-07-09T05:48:42Z)
• Code Less, Align More: Efficient LLM Fine-tuning for Code Generation with Data Pruning [4.975728472540823]
  We present techniques that integrate various clustering and pruning metrics to selectively reduce training data without compromising the accuracy and functionality of the generated code. Our experiments show that these pruning strategies not only reduce the computational resources needed but also enhance the overall quality code generation.
  arXiv  Detail & Related papers  (2024-07-06T10:30:43Z)
• Applying RLAIF for Code Generation with API-usage in Lightweight LLMs [15.366324461797582]
  Reinforcement Learning from AI Feedback (RLAIF) has demonstrated significant potential across various domains. This paper introduces an RLAIF framework for improving the code generation abilities of lightweight (1B parameters) LLMs.
  arXiv  Detail & Related papers  (2024-06-28T17:16:03Z)
• Code Needs Comments: Enhancing Code LLMs with Comment Augmentation [91.52444946362547]
  We introduce a novel data augmentation method that generates comments for existing code, coupled with a data filtering strategy that filters out code data poorly correlated with natural language. We conducted experiments on three code-focused Large Language Models and observed consistent improvements in performance on two widely-used programming skill benchmarks.
  arXiv  Detail & Related papers  (2024-02-20T13:56:38Z)
• Automatic Unit Test Data Generation and Actor-Critic Reinforcement Learning for Code Synthesis [16.88062487980405]
  We present a novel approach to automatically obtain data consisting of function signatures and associated Unit Tests. We show that it, in conjunction with automatically generated training data, leads to improvement of a pre-trained code language model's performance.
  arXiv  Detail & Related papers  (2023-10-20T17:13:16Z)
• Reflection-Tuning: Data Recycling Improves LLM Instruction-Tuning [79.32236399694077]
  Low-quality data in the training set are usually detrimental to instruction tuning. We propose a novel method, termed "reflection-tuning" This approach utilizes an oracle LLM to recycle the original training data by introspecting and enhancing the quality of instructions and responses in the data.
  arXiv  Detail & Related papers  (2023-10-18T05:13:47Z)
• Large Language Models can Contrastively Refine their Generation for Better Sentence Representation Learning [57.74233319453229]
  Large language models (LLMs) have emerged as a groundbreaking technology and their unparalleled text generation capabilities have sparked interest in their application to the fundamental sentence representation learning task. We propose MultiCSR, a multi-level contrastive sentence representation learning framework that decomposes the process of prompting LLMs to generate a corpus. Our experiments reveal that MultiCSR enables a less advanced LLM to surpass the performance of ChatGPT, while applying it to ChatGPT achieves better state-of-the-art results.
  arXiv  Detail & Related papers  (2023-10-17T03:21:43Z)
• MLLM-DataEngine: An Iterative Refinement Approach for MLLM [62.30753425449056]
  We propose a novel closed-loop system that bridges data generation, model training, and evaluation. Within each loop, the MLLM-DataEngine first analyze the weakness of the model based on the evaluation results. For targeting, we propose an Adaptive Bad-case Sampling module, which adjusts the ratio of different types of data. For quality, we resort to GPT-4 to generate high-quality data with each given data type.
  arXiv  Detail & Related papers  (2023-08-25T01:41:04Z)
• Reinforced Self-Training (ReST) for Language Modeling [56.75447441157628]
  Reinforcement learning from human feedback (RLHF) can improve the quality of large language model's (LLM) outputs by aligning them with human preferences. We propose a simple algorithm for aligning LLMs with human preferences inspired by growing batch reinforcement learning (RL), which we call Reinforced Self-Training (ReST) Our results show that ReST can substantially improve translation quality, as measured by automated metrics and human evaluation on machine translation benchmarks in a compute and sample-efficient manner.
  arXiv  Detail & Related papers  (2023-08-17T14:12:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.