Assessing Large Language Models in Generating RTL Design Specifications

• URL: http://arxiv.org/abs/2512.00045v1
• Date: Mon, 17 Nov 2025 10:06:24 GMT
• Title: Assessing Large Language Models in Generating RTL Design Specifications
• Authors: Hung-Ming Huang, Yu-Hsin Yang, Fu-Chieh Chang, Yun-Chia Hsu, Yin-Yu Lin, Ming-Fang Tsai, Chun-Chih Yang, Pei-Yuan Wu,
• Abstract summary: Engineers currently should manually interpret existing RTL code and write specifications, a slow and error-prone process.<n>We investigate how prompting strategies affect RTL-to-specification quality and introduce metrics for faithfully evaluating generated specs.
• Score: 2.4125580419022477
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As IC design grows more complex, automating comprehension and documentation of RTL code has become increasingly important. Engineers currently should manually interpret existing RTL code and write specifications, a slow and error-prone process. Although LLMs have been studied for generating RTL from specifications, automated specification generation remains underexplored, largely due to the lack of reliable evaluation methods. To address this gap, we investigate how prompting strategies affect RTL-to-specification quality and introduce metrics for faithfully evaluating generated specs. We also benchmark open-source and commercial LLMs, providing a foundation for more automated and efficient specification workflows in IC design.

Related papers

• Step-Level Sparse Autoencoder for Reasoning Process Interpretation [48.99201531966593]
  Large Language Models (LLMs) have achieved strong complex reasoning capabilities through Chain-of-Thought (CoT) reasoning.<n>We propose step-level sparse autoencoder (SSAE), which serves as an analytical tool to disentangle different aspects of LLMs' reasoning steps into sparse features.<n> Experiments on multiple base models and reasoning tasks show the effectiveness of the extracted features.
  arXiv  Detail & Related papers  (2026-03-03T14:25:02Z)
• RAVEL: Reasoning Agents for Validating and Evaluating LLM Text Synthesis [78.32151470154422]
  We introduce RAVEL, an agentic framework that enables the testers to autonomously plan and execute typical synthesis operations.<n>We present C3EBench, a benchmark comprising 1,258 samples derived from professional human writings.<n>By augmenting RAVEL with SOTA LLMs as operators, we find that such agentic text synthesis is dominated by the LLM's reasoning capability.
  arXiv  Detail & Related papers  (2026-02-28T14:47:34Z)
• PRO-V: An Efficient Program Generation Multi-Agent System for Automatic RTL Verification [6.983135183126461]
  Pro-V is a fully program generation multi-agent system for robust RTL verification.<n>It incorporates an efficient best-of-n iterative sampling strategy to enhance the correctness of generated testbenches.<n>Pro-V attains a verification accuracy of 87.17% on golden RTL implementations and 76.28% on RTL mutants.
  arXiv  Detail & Related papers  (2025-06-13T20:06:34Z)
• Evaluating Large Language Models for Real-World Engineering Tasks [75.97299249823972]
  This paper introduces a curated database comprising over 100 questions derived from authentic, production-oriented engineering scenarios.<n>Using this dataset, we evaluate four state-of-the-art Large Language Models (LLMs)<n>Our results show that LLMs demonstrate strengths in basic temporal and structural reasoning but struggle significantly with abstract reasoning, formal modeling, and context-sensitive engineering logic.
  arXiv  Detail & Related papers  (2025-05-12T14:05:23Z)
• RTL++: Graph-enhanced LLM for RTL Code Generation [0.0]
  Traditional register transfer level (RTL) design methods are manual, time-consuming, and prone to errors.<n>Open-source models offer alternatives; however, they frequently fall short in quality/correctness.<n>This paper proposes RTL++, a first-of-its-kind LLM-assisted method for RTL code generation.
  arXiv  Detail & Related papers  (2025-05-11T00:17:26Z)
• Self-Steering Language Models [113.96916935955842]
  DisCIPL is a method for "self-steering" language models (LMs)<n>DisCIPL generates a task-specific inference program that is executed by a population of Follower models.<n>Our work opens up a design space of highly-parallelized Monte Carlo inference strategies.
  arXiv  Detail & Related papers  (2025-04-09T17:54:22Z)
• TuRTLe: A Unified Evaluation of LLMs for RTL Generation [0.6010802600885173]
  We propose TuRTLe, a unified evaluation framework designed to assess LLMs across key RTL generation tasks.<n>We benchmark a diverse set of open LLMs and analyze their strengths and weaknesses in EDA-specific tasks.<n>Our results show that reasoning-based models, such as DeepSeek R1, consistently outperform others across multiple evaluation criteria.
  arXiv  Detail & Related papers  (2025-03-31T07:43:12Z)
• Exploring Code Language Models for Automated HLS-based Hardware Generation: Benchmark, Infrastructure and Analysis [14.458529723566379]
  Large language models (LLMs) can be employed for programming languages such as Python and C++.<n>This paper explores leveraging LLMs to generate High-Level Synthesis (HLS)-based hardware design.
  arXiv  Detail & Related papers  (2025-02-19T17:53:59Z)
• RTLSquad: Multi-Agent Based Interpretable RTL Design [3.1734541757969463]
  Large Language Models (LLMs) offer new approaches for automatic RTL code generation and optimization.<n>To address this, we propose RTLSquad, a novel LLM-Based Multi-Agent system for interpretable RTL code generation.
  arXiv  Detail & Related papers  (2025-01-06T02:57:54Z)
• Reference Trustable Decoding: A Training-Free Augmentation Paradigm for Large Language Models [79.41139393080736]
  Large language models (LLMs) have rapidly advanced and demonstrated impressive capabilities. In-Context Learning (ICL) and. Efficient Fine-Tuning (PEFT) are currently two mainstream methods for augmenting. LLMs to downstream tasks. We propose Reference Trustable Decoding (RTD), a paradigm that allows models to quickly adapt to new tasks without fine-tuning.
  arXiv  Detail & Related papers  (2024-09-30T10:48:20Z)
• OriGen:Enhancing RTL Code Generation with Code-to-Code Augmentation and Self-Reflection [54.775409528658486]
  OriGen is a fully open-source framework that incorporates self-reflection capabilities and a novel dataset augmentation methodology. Our approach employs a code-tocode augmentation technique to enhance the quality of open-source RTL code datasets.
  arXiv  Detail & Related papers  (2024-07-23T07:22:25Z)
• LLM4EDA: Emerging Progress in Large Language Models for Electronic Design Automation [74.7163199054881]
  Large Language Models (LLMs) have demonstrated their capability in context understanding, logic reasoning and answer generation. We present a systematic study on the application of LLMs in the EDA field. We highlight the future research direction, focusing on applying LLMs in logic synthesis, physical design, multi-modal feature extraction and alignment of circuits.
  arXiv  Detail & Related papers  (2023-12-28T15:09:14Z)
• RTLLM: An Open-Source Benchmark for Design RTL Generation with Large Language Model [6.722151433412209]
  We propose an open-source benchmark named RTLLM, for generating design RTL with natural language instructions. This benchmark can automatically provide a quantitative evaluation of any given LLM-based solution. We also propose an easy-to-use yet surprisingly effective prompt engineering technique named self-planning.
  arXiv  Detail & Related papers  (2023-08-10T05:24:41Z)
• Certified Reinforcement Learning with Logic Guidance [78.2286146954051]
  We propose a model-free RL algorithm that enables the use of Linear Temporal Logic (LTL) to formulate a goal for unknown continuous-state/action Markov Decision Processes (MDPs) The algorithm is guaranteed to synthesise a control policy whose traces satisfy the specification with maximal probability.
  arXiv  Detail & Related papers  (2019-02-02T20:09:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.