HLS-Seek: QoR-Aware Code Generation for High-Level Synthesis via Proxy Comparative Reward Reinforcement Learning

• URL: http://arxiv.org/abs/2605.13536v1
• Date: Wed, 13 May 2026 13:47:03 GMT
• Title: HLS-Seek: QoR-Aware Code Generation for High-Level Synthesis via Proxy Comparative Reward Reinforcement Learning
• Abstract summary: textbfHLS-Seek replaces expensive synthesis-in-the-loop RL with a comparative proxy reward model.<n>HLS-Seek achieves 81.5% syntax correctness pass@1 and 81.4% Func@5 on HLS-eval with only 7B parameters.
• Score: 34.436059306800765
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: High-Level Synthesis (HLS) compiles algorithmic C/C++ descriptions into hardware, with Quality of Results (QoR) -- latency and resource utilization -- critically governed by pragma configurations and code structure. Existing LLM-based HLS approaches train for functional correctness but ignore QoR entirely. We observe that reinforcement learning (RL) for HLS does not require absolute synthesis results -- only relative comparisons between candidates. Based on this insight, we propose \textbf{HLS-Seek}, a QoR-aware NL-to-HLS framework that replaces expensive synthesis-in-the-loop RL with a comparative proxy reward model achieving 99.53\% Pareto-dominance accuracy. To prevent reward hacking, we introduce \textit{uncertainty-aware Monte Carlo (MC) dropout switching} that selectively invokes real Vitis HLS synthesis for low-confidence candidates and online updates the proxy, creating a self-improving reward system. HLS-Seek achieves 81.5\% syntax correctness pass@1 and 81.4\% Func@5 on HLS-eval with only 7B parameters, surpassing GPT-5.1 and other frontier models while achieving 8.5$\times$ faster training than real-reward RL. On QoR evaluation, HLS-Seek achieves the lowest latency on 16/30 kernels and Pareto-dominates HLS-specific baselines on 9 kernels.

Related papers

• CVeDRL: An Efficient Code Verifier via Difficulty-aware Reinforcement Learning [57.24524263804788]
  Code verifiers play a critical role in post-verification for LLM-based code generation.<n>Existing supervised fine-tuning methods suffer from data scarcity, high failure rates, and poor inference efficiency.<n>We show that naive RL with only functionality rewards fails to generate effective unit tests for difficult branches and samples.
  arXiv  Detail & Related papers  (2026-01-30T10:33:29Z)
• Bench4HLS: End-to-End Evaluation of LLMs in High-Level Synthesis Code Generation [0.0]
  Large language models (LLMs) have shown strong capabilities in code generation, including hardware design at register-transfer level (RTL)<n>The ratio of HLS to RTL-focused studies has shifted from 1:10 to 2:10 in the past six months.<n>This growing trend highlights the need for a comprehensive benchmarking and evaluation framework dedicated to LLM-based HLS.
  arXiv  Detail & Related papers  (2026-01-16T20:52:42Z)
• QeRL: Beyond Efficiency -- Quantization-enhanced Reinforcement Learning for LLMs [80.76334908639745]
  We propose QeRL, a Quantization-enhanced Reinforcement Learning framework for large language models (LLMs)<n>QeRL addresses issues by combining NVFP4 quantization with Low-Rank Adaptation (LoRA)<n>Experiments demonstrate that QeRL delivers over 1.5 times speedup in the rollout phase.
  arXiv  Detail & Related papers  (2025-10-13T17:55:09Z)
• TimelyHLS: LLM-Based Timing-Aware and Architecture-Specific FPGA HLS Optimization [0.0]
  TimelyHLS generates HLS code annotated with both timing-critical and design-specific pragmas.<n>TimelyHLS consistently achieves timing closure and functional correctness across platforms.
  arXiv  Detail & Related papers  (2025-07-23T22:08:15Z)
• HLSTester: Efficient Testing of Behavioral Discrepancies with LLMs for High-Level Synthesis [4.544714097232974]
  Existing methods for testing behavioral discrepancies in high-level synthesis (HLS) are still immature.<n>We propose HLSTester, a large language model (LLM) aided testing framework that efficiently detects behavioral discrepancies in HLS.<n>To mitigate hallucinations in LLMs, the testbenches for original C/C++ programs are leveraged to guide LLMs in generating HLS-compatible testbenches.
  arXiv  Detail & Related papers  (2025-04-20T14:45:01Z)
• HLS-Eval: A Benchmark and Framework for Evaluating LLMs on High-Level Synthesis Design Tasks [4.71707720395444]
  We introduce HLS-Eval, the first complete benchmark and evaluation framework for HLS-driven design.<n>The benchmark includes 94 unique designs drawn from standard HLS benchmarks and novel sources.<n>Beyond the benchmark, HLS-Eval offers a modular Python framework for automated, parallel evaluation of both local and hosted LLMs.
  arXiv  Detail & Related papers  (2025-04-16T17:30:36Z)
• Hint-before-Solving Prompting: Guiding LLMs to Effectively Utilize Encoded Knowledge [85.17343729885003]
  We introduce Hint-before-Solving Prompting (HSP), which guides the model to generate hints for solving the problem. HSP can effectively improve the accuracy of reasoning tasks. We build the HSPMATH dataset based on HSP and fine-tuned Llemma-7B, reaching 64.3 accuracy.
  arXiv  Detail & Related papers  (2024-02-22T05:58:03Z)
• Hierarchical Source-to-Post-Route QoR Prediction in High-Level Synthesis with GNNs [25.920672727699984]
  We propose a hierarchical post-route QoR prediction approach for FPGA HLS. By adopting our proposed methodology, the runtime for design space exploration in HLS is shortened to tens of minutes.
  arXiv  Detail & Related papers  (2024-01-14T07:24:08Z)
• RRHF: Rank Responses to Align Language Models with Human Feedback without tears [69.68672043223249]
  InstructGPT implements RLHF through several stages, including Supervised Fine-Tuning (SFT), reward model training, and Proximal Policy Optimization (PPO) We propose a novel learning paradigm called RRHF, which scores sampled responses from different sources via a logarithm of conditional probabilities. We evaluate RRHF on the Helpful and Harmless dataset, demonstrating comparable alignment performance with PPO by reward model score and human labeling.
  arXiv  Detail & Related papers  (2023-04-11T15:53:40Z)
• LCRL: Certified Policy Synthesis via Logically-Constrained Reinforcement Learning [78.2286146954051]
  LCRL implements model-free Reinforcement Learning (RL) algorithms over unknown Decision Processes (MDPs) We present case studies to demonstrate the applicability, ease of use, scalability, and performance of LCRL.
  arXiv  Detail & Related papers  (2022-09-21T13:21:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.