LLM Agents Perform Controlled Experiments Using Simulation Models

• URL: http://arxiv.org/abs/2608.23622v1
• Date: Sat, 22 Aug 2026 18:54:52 GMT
• Title: LLM Agents Perform Controlled Experiments Using Simulation Models
• Abstract summary: Large language models (LLMs) have shown strong capabilities in reasoning, planning, and tool use.<n>We propose a multi-agent framework that enables LLM agents to conduct controlled experiments with scientific simulation models for pharmaceutical process design.
• Score: 1.0449796527610344
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) have shown strong capabilities in reasoning, planning, and tool use, but many scientific and engineering tasks require more than plausible text and code generation. They require understanding how a system responds to intervention, which in practice depends on controlled experimentation. In this work, we propose a multi-agent framework that enables LLM agents to conduct controlled experiments with scientific simulation models for pharmaceutical process design. Given a user query and a baseline configuration, the system constructs a structured task representation, designs experiments, executes comparative simulation, interprets the resulting outcomes, and synthesizes evidence-based recommendations for process parameter optimization. By coupling language models with high-fidelity simulation models in an interactive agent framework, the proposed system supports reasoning through intervention, comparison, and observation. As a result, it produces more specific and actionable outputs than language-only reasoning. In an industrial application setting, this advantage is reflected in higher output specificity as well as improved user-rated correctness and helpfulness. Ablation studies and visualized case analyses further demonstrate the effectiveness and practical utility of simulation-integrated experimental reasoning.

Related papers

• Simulate, Reason, Decide: Scientific Reasoning with LLMs for Simulation-Driven Decision Making [50.05403024674141]
  MechSim is a mechanism-grounded neuro-symbolic reasoning framework for executable scientific simulators.<n>Our framework represents simulators through a shared structured schema capturing assumptions, variables, mechanism dependencies, and execution traces.<n>We evaluate our approach across multiple high-stakes domains and show that it improves mechanism-level explanation quality, simulator analysis, and downstream decision-making reliability.
  arXiv  Detail & Related papers  (2026-06-03T06:36:51Z)
• Multi-Agent Specification-based Metamorphic Testing of FMU-Based Simulations [1.58310730488265]
  In industrial domains, Functional Mock-up Interface (FMI) is used to exchange simulation models as Functional Mock-up Units (FMUs)<n> deriving effective test oracles for these simulation models remains challenging due to the absence of explicit expected outputs.<n>We propose an LLM-powered multi-agent workflow for specification-based metamorphic testing of FMU-based simulation models.
  arXiv  Detail & Related papers  (2026-05-24T14:30:56Z)
• GRACE: an Agentic AI for Particle Physics Experiment Design and Simulation [0.0]
  GRACE is a simulation-native agent for autonomous experimental design in high-energy and nuclear physics.<n>It autonomously explores design modifications using first-principles Monte Carlo methods.<n>It evaluates candidate designs through repeated simulation, physics-motivated utility functions, and budget-aware escalation.
  arXiv  Detail & Related papers  (2026-01-31T01:12:55Z)
• Large Language Model Agent for User-friendly Chemical Process Simulations [0.0]
  A large language model (LLM) agent is integrated with AVEVA Process Model Protocol (MCP), allowing natural language simulations.<n>Two case studies assess the framework across different task complexities and interaction modes.<n>The framework benefits both educational purposes, by translating technical concepts and demonstrating, and experienced practitioners by automating data extraction, speeding routine tasks, and supporting.<n>While current limitations such as oversimplification, calculation errors, and technical hiccups mean expert oversight is still needed, the framework suggests LLM-based agents can become valuable collaborators.
  arXiv  Detail & Related papers  (2026-01-15T12:18:45Z)
• An Agentic Framework for Autonomous Materials Computation [70.24472585135929]
  Large Language Models (LLMs) have emerged as powerful tools for accelerating scientific discovery.<n>Recent advances integrate LLMs into agentic frameworks, enabling retrieval, reasoning, and tool use for complex scientific experiments.<n>Here, we present a domain-specialized agent designed for reliable automation of first-principles materials computations.
  arXiv  Detail & Related papers  (2025-12-22T15:03:57Z)
• Sample-Efficient Online Learning in LM Agents via Hindsight Trajectory Rewriting [92.57796055887995]
  We introduce ECHO, a prompting framework that adapts hindsight experience replay from reinforcement learning for language model agents.<n> ECHO generates optimized trajectories for alternative goals that could have been achieved during failed attempts.<n>We evaluate ECHO on stateful versions of XMiniGrid, a text-based navigation and planning benchmark, and PeopleJoinQA, a collaborative information-gathering enterprise simulation.
  arXiv  Detail & Related papers  (2025-10-11T18:11:09Z)
• MAPS: Advancing Multi-Modal Reasoning in Expert-Level Physical Science [62.96434290874878]
  Current Multi-Modal Large Language Models (MLLM) have shown strong capabilities in general visual reasoning tasks.<n>We develop a new framework, named Multi-Modal Scientific Reasoning with Physics Perception and Simulation (MAPS) based on an MLLM.<n>MAPS decomposes expert-level multi-modal reasoning task into physical diagram understanding via a Physical Perception Model (PPM) and reasoning with physical knowledge via a simulator.
  arXiv  Detail & Related papers  (2025-01-18T13:54:00Z)
• Designing Reliable Experiments with Generative Agent-Based Modeling: A Comprehensive Guide Using Concordia by Google DeepMind [39.96801170116895]
  Generative Agent-Based Modeling (GABM) offers a solution by enabling scholars to create simulations where AI-driven agents can generate complex behaviors. This paper introduces a framework for designing reliable experiments using GABM, making sophisticated simulation techniques more accessible to researchers across various fields.
  arXiv  Detail & Related papers  (2024-11-11T14:45:08Z)
• LLM experiments with simulation: Large Language Model Multi-Agent System for Simulation Model Parametrization in Digital Twins [4.773175285216063]
  This paper presents a novel framework that applies large language models (LLMs) to automate the parametrization of simulation models in digital twins. The proposed approach enhances the usability of simulation model by infusing it with knowledges from LLM. The system has the potential to increase user-friendliness and reduce the cognitive load on human users.
  arXiv  Detail & Related papers  (2024-05-28T11:59:40Z)
• Unlocking the Potential of User Feedback: Leveraging Large Language Model as User Simulator to Enhance Dialogue System [65.93577256431125]
  We propose an alternative approach called User-Guided Response Optimization (UGRO) to combine it with a smaller task-oriented dialogue model. This approach uses LLM as annotation-free user simulator to assess dialogue responses, combining them with smaller fine-tuned end-to-end TOD models. Our approach outperforms previous state-of-the-art (SOTA) results.
  arXiv  Detail & Related papers  (2023-06-16T13:04:56Z)
• Synthetic Data-Based Simulators for Recommender Systems: A Survey [55.60116686945561]
  This survey aims at providing a comprehensive overview of the recent trends in the field of modeling and simulation. We start with the motivation behind the development of frameworks implementing the simulations -- simulators. We provide a new consistent classification of existing simulators based on their functionality, approbation, and industrial effectiveness.
  arXiv  Detail & Related papers  (2022-06-22T19:33:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.