Reliable LLM-Generated Programs for High-Energy Physics Experiments through Graph-Grounded Software Knowledge

• URL: http://arxiv.org/abs/2609.01095v1
• Date: Tue, 01 Sep 2026 11:35:02 GMT
• Title: Reliable LLM-Generated Programs for High-Energy Physics Experiments through Graph-Grounded Software Knowledge
• Abstract summary: General-purpose large language models (LLMs) often produce unreliable programs for such tasks.<n>We organize these software relations before generation and retrieve task-relevant knowledge at inference time.<n>We evaluate a complete grounding system that combines hybrid retrieval over a heterogeneous software knowledge graph.
• Score: 18.431539281285225
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Extracting physics information from modern particle-physics experiments requires multistage analyses implemented on top of large and highly interconnected software ecosystems. General-purpose large language models (LLMs) often produce unreliable programs for such tasks because a user request alone rarely specifies the required APIs, dependencies, and usage conventions. We organize these software relations before generation and retrieve task-relevant knowledge at inference time. Using the open-source ROOT framework as a representative and reproducible testbed, we evaluate a complete grounding system that combines hybrid retrieval over a heterogeneous software knowledge graph, skill-selected workflow examples, and execution-guided repair. On a benchmark of 275 ROOT tasks, grounding improves first-attempt execution from 58.5% to 76.0% under Claude Code orchestration and from 51.3% to 64.0% under standalone orchestration. Final success increases from 90.5% to 96.0% and from 78.9% to 90.9%, respectively, while the average generation cost per successful task increases by only 1.3% and 3.2%. The gains persist under a strong coding agent, indicating that explicit software knowledge remains valuable even when agentic scaffolding is already in place. Because the method captures software relations common to large codebases rather than facts specific to ROOT or a particular model, it should transfer to other experiment frameworks and proprietary software, especially where documentation is sparse or internal dependencies are complex.

Related papers

• Orchard: An Open-Source Agentic Modeling Framework [124.68499958175111]
  We present Orchard, an open-source framework for scalable agentic modeling.<n>At its core is Orchard Env, a lightweight environment service providing reusable primitives for sandbox lifecycle management.<n>On top of Orchard Env, we build three agentic modeling recipes.
  arXiv  Detail & Related papers  (2026-05-14T16:35:12Z)
• Workspace-Bench 1.0: Benchmarking AI Agents on Workspace Tasks with Large-Scale File Dependencies [29.138677906501865]
  We introduce Workspace-Bench, a benchmark for evaluating AI agents on Workspace Learning involving Large-Scale File Dependencies.<n>We construct realistic workspaces with 5 worker profiles, 74 file types, 20,476 files (up to 20GB) and curate 388 tasks, each with its own file dependency graph, evaluated across 7,399 total rubrics.
  arXiv  Detail & Related papers  (2026-05-05T10:17:06Z)
• Agentic AI in the Software Development Lifecycle: Architecture, Empirical Evidence, and the Reshaping of Software Engineering [0.0]
  Modern agentic systems operate at the granularity of a repository, a feature, or an algorithm.<n>We propose a six-layer reference architecture for agentic software engineering systems.<n>We argue that the central object of inquiry has shifted from code generation to delegated execution under human supervision.
  arXiv  Detail & Related papers  (2026-04-29T04:06:47Z)
• Revisiting Quantum Code Generation: Where Should Domain Knowledge Live? [0.0]
  We study specialization strategies for Qiskit code generation using the Qiskit-HumanEval benchmark.<n>We show that modern general-purpose LLMs consistently outperform the parameter-specialized baseline.
  arXiv  Detail & Related papers  (2026-03-23T16:46:39Z)
• SciAgentGym: Benchmarking Multi-Step Scientific Tool-use in LLM Agents [100.12367115920121]
  We introduce SciGymAgent, a scalable interactive environment featuring 1,780 domain-specific tools across four natural science disciplines.<n>We also present SciAgentBench, a tiered evaluation suite designed to stress-test agentic capabilities.
  arXiv  Detail & Related papers  (2026-02-13T14:58:18Z)
• SWE-RM: Execution-free Feedback For Software Engineering Agents [61.86380395896069]
  Execution-based feedback is widely used in the development of coding agents through test-time scaling (TTS) and reinforcement learning (RL)<n>In contrast, execution-free feedback from reward models can provide more fine-grained signals without depending on unit test cases.<n>We introduce SWE-RM, an accurate and robust reward model adopting a mixture-of-experts architecture with 30B total parameters and 3B activated during inference.
  arXiv  Detail & Related papers  (2025-12-26T08:26:18Z)
• Lifecycle-Aware code generation: Leveraging Software Engineering Phases in LLMs [12.70863561286374]
  We introduce a lifecycle-aware framework that incorporates intermediate artifacts into both the training and inference stages.<n> Experiments show that lifecycle-level fine-tuning improves code correctness by up to 75% over the same model before fine-tuning.<n>Open-source LLMs, once fine-tuned under our framework, match or slightly outperform models pretrained on code.
  arXiv  Detail & Related papers  (2025-10-28T02:54:02Z)
• OmniEAR: Benchmarking Agent Reasoning in Embodied Tasks [52.87238755666243]
  We present OmniEAR, a framework for evaluating how language models reason about physical interactions, tool usage, and multi-agent coordination in embodied tasks.<n>We model continuous physical properties and complex spatial relationships across 1,500 scenarios spanning household and industrial domains.<n>Our systematic evaluation reveals severe performance degradation when models must reason from constraints.
  arXiv  Detail & Related papers  (2025-08-07T17:54:15Z)
• Skywork-SWE: Unveiling Data Scaling Laws for Software Engineering in LLMs [19.766885088032932]
  Software engineering (SWE) has emerged as a crucial testbed for next-generation LLM agents.<n>Most existing datasets are limited to only a few thousand GitHub-sourced instances.<n>We propose an incremental, automated data-curation pipeline that systematically scales both the volume and diversity of SWE datasets.
  arXiv  Detail & Related papers  (2025-06-24T03:53:36Z)
• Advancing LLM Reasoning Generalists with Preference Trees [119.57169648859707]
  We introduce Eurus, a suite of large language models (LLMs) optimized for reasoning. Eurus models achieve state-of-the-art results among open-source models on a diverse set of benchmarks.
  arXiv  Detail & Related papers  (2024-04-02T16:25:30Z)
• SOEN-101: Code Generation by Emulating Software Process Models Using Large Language Model Agents [50.82665351100067]
  FlowGen is a code generation framework that emulates software process models based on multiple Large Language Model (LLM) agents. We evaluate FlowGenScrum on four benchmarks: HumanEval, HumanEval-ET, MBPP, and MBPP-ET.
  arXiv  Detail & Related papers  (2024-03-23T14:04:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.