OriGen:Enhancing RTL Code Generation with Code-to-Code Augmentation and Self-Reflection

• URL: http://arxiv.org/abs/2407.16237v2
• Date: Mon, 2 Sep 2024 07:25:21 GMT
• Title: OriGen:Enhancing RTL Code Generation with Code-to-Code Augmentation and Self-Reflection
• Authors: Fan Cui, Chenyang Yin, Kexing Zhou, Youwei Xiao, Guangyu Sun, Qiang Xu, Qipeng Guo, Demin Song, Dahua Lin, Xingcheng Zhang, Yun, Liang,
• Abstract summary: OriGen is a fully open-source framework that incorporates self-reflection capabilities and a novel dataset augmentation methodology. Our approach employs a code-tocode augmentation technique to enhance the quality of open-source RTL code datasets.
• Score: 54.775409528658486
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Recent studies have demonstrated the significant potential of Large Language Models (LLMs) in generating Register Transfer Level (RTL) code, with notable advancements showcased by commercial models such as GPT-4 and Claude3-Opus. However, these proprietary LLMs often raise concerns regarding privacy and security. While open-source LLMs offer solutions to these concerns, they typically underperform commercial models in RTL code generation tasks, primarily due to the scarcity of high-quality open-source RTL datasets. To address this challenge, we introduce OriGen , a fully open-source framework that incorporates self-reflection capabilities and a novel dataset augmentation methodology for generating high-quality, large-scale RTL code. Our approach employs a code-tocode augmentation technique to enhance the quality of open-source RTL code datasets. Furthermore, OriGen can rectify syntactic errors through a self-reflection process that leverages compiler feedback. Experimental results demonstrate that OriGen significantly outperforms other open-source alternatives in RTL code generation. It surpasses the previous best-performing open-source LLM by 12.8% and even exceeds GPT-4 Turbo in the pass@1 metric on the VerilogEval-Human benchmark. Moreover, OriGen exhibits superior capabilities in self-reflection and error correction, outperforming GPT-4 by 19.9% on a benchmark designed to evaluate self-reflection capabilities.

Related papers

• Optimizing Knowledge Integration in Retrieval-Augmented Generation with Self-Selection [72.92366526004464]
  Retrieval-Augmented Generation (RAG) has proven effective in enabling Large Language Models (LLMs) to produce more accurate and reliable responses. We propose a novel Self-Selection RAG framework, where the LLM is made to select from pairwise responses generated with internal parametric knowledge solely.
  arXiv  Detail & Related papers  (2025-02-10T04:29:36Z)
• PromptV: Leveraging LLM-powered Multi-Agent Prompting for High-quality Verilog Generation [9.990225157705966]
  This paper proposes a novel multi-agent prompt learning framework to address limitations and enhance code generation quality. We show for the first time that multi-agent architectures can effectively mitigate the degeneration risk while improving code error correction capabilities.
  arXiv  Detail & Related papers  (2024-12-15T01:58:10Z)
• EDA-Aware RTL Generation with Large Language Models [0.7831852829409273]
  Large Language Models (LLMs) have become increasingly popular for generating RTL code. producing error-free RTL code in a zero-shot setting remains highly challenging for even state-of-the-art LLMs. We introduce AIvril2, a self-verifying, LLM-agnostic agentic framework aimed at enhancing RTL code generation through iterative corrections of both syntax and functional errors.
  arXiv  Detail & Related papers  (2024-11-21T00:37:51Z)
• OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models [70.72097493954067]
  Large language models (LLMs) for code have become indispensable in various domains, including code generation, reasoning tasks and agent systems. While open-access code LLMs are increasingly approaching the performance levels of proprietary models, high-quality code LLMs remain limited. We introduce OpenCoder, a top-tier code LLM that not only achieves performance comparable to leading models but also serves as an "open cookbook" for the research community.
  arXiv  Detail & Related papers  (2024-11-07T17:47:25Z)
• AIvril: AI-Driven RTL Generation With Verification In-The-Loop [0.7831852829409273]
  Large Language Models (LLMs) are computational models capable of performing complex natural language processing tasks. This paper introduces AIvril, a framework designed to enhance the accuracy and reliability of RTL-aware LLMs.
  arXiv  Detail & Related papers  (2024-09-03T15:07:11Z)
• AutoVCoder: A Systematic Framework for Automated Verilog Code Generation using LLMs [27.179391677757565]
  We develop AutoVCoder, a framework that significantly improves the correctness of generating Verilog code. Our framework integrates three novel techniques, including a high-quality hardware dataset generation approach. AutoVCoder shows a 0.5% and 2.2% improvement in functional correctness on the EvalMachine and EvalHuman benchmarks compared with BetterV.
  arXiv  Detail & Related papers  (2024-07-21T16:42:45Z)
• UniGen: A Unified Framework for Textual Dataset Generation Using Large Language Models [88.16197692794707]
  UniGen is a comprehensive framework designed to produce diverse, accurate, and highly controllable datasets. To augment data diversity, UniGen incorporates an attribute-guided generation module and a group checking feature. Extensive experiments demonstrate the superior quality of data generated by UniGen.
  arXiv  Detail & Related papers  (2024-06-27T07:56:44Z)
• DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoning [56.887047551101574]
  We present DS-Agent, a novel framework that harnesses large language models (LLMs) agent and case-based reasoning (CBR) In the development stage, DS-Agent follows the CBR framework to structure an automatic iteration pipeline, which can flexibly capitalize on the expert knowledge from Kaggle. In the deployment stage, DS-Agent implements a low-resource deployment stage with a simplified CBR paradigm, significantly reducing the demand on foundational capabilities of LLMs.
  arXiv  Detail & Related papers  (2024-02-27T12:26:07Z)
• StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback [58.20547418182074]
  We introduce StepCoder, a novel framework for code generation, consisting of two main components. CCCS addresses the exploration challenge by breaking the long sequences code generation task into a Curriculum of Code Completion Subtasks. FGO only optimize the model by masking the unexecuted code segments to provide Fine-Grained Optimization. Our method improves the ability to explore the output space and outperforms state-of-the-art approaches in corresponding benchmarks.
  arXiv  Detail & Related papers  (2024-02-02T13:14:31Z)
• Evaluating Diverse Large Language Models for Automatic and General Bug Reproduction [12.851941377433285]
  Large language models (LLMs) have been demonstrated to be adept at natural language processing and code generation. Our proposed technique LIBRO could successfully reproduce about one-third of all bugs in the widely used Defects4J benchmark.
  arXiv  Detail & Related papers  (2023-11-08T08:42:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.