HLSFactory-Agent: Large-Scale Agentic HLS Dataset Construction from Academic and Open-Source Projects

• URL: http://arxiv.org/abs/2609.09519v1
• Date: Tue, 08 Sep 2026 23:02:07 GMT
• Title: HLSFactory-Agent: Large-Scale Agentic HLS Dataset Construction from Academic and Open-Source Projects
• Abstract summary: Building large, diverse datasets of high-level synthesis (HLS) designs remains an open challenge.<n>We present HLSFactory-Agent, an LLM agent that automates large-scale HLS dataset curation.<n>It runs the open-source Pi agent framework inside Docker containers to build and evaluate each extracted design.<n>We provide open-source scripts to scrape and index papers from computer architecture, EDA, and FPGA conferences.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Building large, diverse datasets of high-level synthesis (HLS) designs beyond common community benchmarks remains an open challenge. This challenge is made urgent by the rise of deep learning and LLMs for hardware design, which demand such datasets to train QoR models and benchmark LLMs on HLS tasks. Despite ongoing efforts to broaden sources, dataset curation still depends on manual work: locating HLS designs across academic publications and open source, then extracting standalone designs from larger codebases. The process is error-prone and demands expert knowledge, iterative testing, and substantial per-repository engineering. To address this, we present HLSFactory-Agent, an LLM agent that automates large-scale HLS dataset curation by extracting standalone designs from larger codebases. HLSFactory-Agent runs the open-source Pi agent framework inside Docker containers to build and evaluate each extracted design. This turnkey automation allows users to pass a GitHub link or code directory to HLSFactory-Agent and receive a folder of extracted HLS designs ready to be integrated into the HLSFactory dataset framework. Additionally, we provide open-source scripts to scrape and index papers from computer architecture, EDA, and FPGA conferences that possibly implement or use HLS designs, allowing for faster human discovery and curation of HLS designs for HLSFactory-Agent. We report initial results from running HLSFactory-Agent across a small subset of our indexed repositories, demonstrating successful extraction of synthesizable designs from structured codebases. We open source HLSFactory-Agent and indexing scripts at https://github.com/sharc-lab/hlsfactory-agent.

Related papers

• RAVEL: Reasoning Agents for Validating and Evaluating LLM Text Synthesis [78.32151470154422]
  We introduce RAVEL, an agentic framework that enables the testers to autonomously plan and execute typical synthesis operations.<n>We present C3EBench, a benchmark comprising 1,258 samples derived from professional human writings.<n>By augmenting RAVEL with SOTA LLMs as operators, we find that such agentic text synthesis is dominated by the LLM's reasoning capability.
  arXiv  Detail & Related papers  (2026-02-28T14:47:34Z)
• Immersion in the GitHub Universe: Scaling Coding Agents to Mastery [60.359983359258955]
  ScaleSWE is an automated, sandboxed multi agent workflow designed to construct high quality SWE data at scale.<n>The system coordinates three specialized agents for environment setup, test creation, and problem description synthesis to process 6 million pull requests across 5200 repositories.
  arXiv  Detail & Related papers  (2026-02-10T15:30:19Z)
• Bench4HLS: End-to-End Evaluation of LLMs in High-Level Synthesis Code Generation [0.0]
  Large language models (LLMs) have shown strong capabilities in code generation, including hardware design at register-transfer level (RTL)<n>The ratio of HLS to RTL-focused studies has shifted from 1:10 to 2:10 in the past six months.<n>This growing trend highlights the need for a comprehensive benchmarking and evaluation framework dedicated to LLM-based HLS.
  arXiv  Detail & Related papers  (2026-01-16T20:52:42Z)
• LM-Searcher: Cross-domain Neural Architecture Search with LLMs via Unified Numerical Encoding [55.5535016040221]
  LM-Searcher is a novel framework for cross-domain neural architecture optimization.<n>Central to our approach is NCode, a universal numerical string representation for neural architectures.<n>Our dataset, encompassing a wide range of architecture-performance pairs, encourages robust and transferable learning.
  arXiv  Detail & Related papers  (2025-09-06T09:26:39Z)
• ForgeHLS: A Large-Scale, Open-Source Dataset for High-Level Synthesis [13.87691887333415]
  We introduce ForgeHLS, a large-scale, open-source dataset explicitly designed for machine learning (ML)-driven HLS research.<n> ForgeHLS comprises over 400k diverse designs generated from 846 kernels covering a broad range of application domains.<n>Compared to existing datasets, ForgeHLS significantly enhances scale, diversity, and design coverage.
  arXiv  Detail & Related papers  (2025-07-04T02:23:46Z)
• HLS-Eval: A Benchmark and Framework for Evaluating LLMs on High-Level Synthesis Design Tasks [4.71707720395444]
  We introduce HLS-Eval, the first complete benchmark and evaluation framework for HLS-driven design.<n>The benchmark includes 94 unique designs drawn from standard HLS benchmarks and novel sources.<n>Beyond the benchmark, HLS-Eval offers a modular Python framework for automated, parallel evaluation of both local and hosted LLMs.
  arXiv  Detail & Related papers  (2025-04-16T17:30:36Z)
• Exploring Code Language Models for Automated HLS-based Hardware Generation: Benchmark, Infrastructure and Analysis [14.458529723566379]
  Large language models (LLMs) can be employed for programming languages such as Python and C++.<n>This paper explores leveraging LLMs to generate High-Level Synthesis (HLS)-based hardware design.
  arXiv  Detail & Related papers  (2025-02-19T17:53:59Z)
• OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models [76.59316249991657]
  Large language models (LLMs) for code have become indispensable in various domains, including code generation, reasoning tasks and agent systems.<n>While open-access code LLMs are increasingly approaching the performance levels of proprietary models, high-quality code LLMs remain limited.<n>We introduce OpenCoder, a top-tier code LLM that not only achieves performance comparable to leading models but also serves as an "open cookbook" for the research community.
  arXiv  Detail & Related papers  (2024-11-07T17:47:25Z)
• Small Agent Can Also Rock! Empowering Small Language Models as Hallucination Detector [114.88975874411142]
  Hallucination detection is a challenging task for large language models (LLMs) We propose an autonomous LLM-based agent framework, called HaluAgent. In HaluAgent, we integrate the LLM, multi-functional toolbox, and design a fine-grained three-stage detection framework.
  arXiv  Detail & Related papers  (2024-06-17T07:30:05Z)
• HLSFactory: A Framework Empowering High-Level Synthesis Datasets for Machine Learning and Beyond [3.206764939601044]
  Machine learning (ML) techniques have been applied to high-level synthesis (HLS) flows for quality-of-result (QoR) prediction and design space exploration (DSE)<n>The scarcity of high-quality HLS datasets and the complexity of building such datasets present challenges.<n>We introduce HLSFactory, a comprehensive framework designed to facilitate the curation and generation of high-quality HLS design datasets.
  arXiv  Detail & Related papers  (2024-05-01T19:02:18Z)
• ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code [76.84199699772903]
  ML-Bench is a benchmark rooted in real-world programming applications that leverage existing code repositories to perform tasks. To evaluate both Large Language Models (LLMs) and AI agents, two setups are employed: ML-LLM-Bench for assessing LLMs' text-to-code conversion within a predefined deployment environment, and ML-Agent-Bench for testing autonomous agents in an end-to-end task execution within a Linux sandbox environment.
  arXiv  Detail & Related papers  (2023-11-16T12:03:21Z)
• CodeTF: One-stop Transformer Library for State-of-the-art Code LLM [72.1638273937025]
  We present CodeTF, an open-source Transformer-based library for state-of-the-art Code LLMs and code intelligence. Our library supports a collection of pretrained Code LLM models and popular code benchmarks. We hope CodeTF is able to bridge the gap between machine learning/generative AI and software engineering.
  arXiv  Detail & Related papers  (2023-05-31T05:24:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.