PACE-Bench: Benchmarking Physics Adaptation via Code Evolution in Dynamic Environments

• URL: http://arxiv.org/abs/2608.14441v1
• Date: Fri, 14 Aug 2026 16:25:43 GMT
• Title: PACE-Bench: Benchmarking Physics Adaptation via Code Evolution in Dynamic Environments
• Abstract summary: PACE-Bench is a simulator-grounded benchmark of 144 source-to-target adaptation pairs across six physics domains.<n>We show that simulator-grounded reflection is more reliable than self-revision.<n>Even revealing exact physical changes does not raise the performance ceiling.
• Score: 8.552837617531766
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Self-evolving agents improve future behavior from interaction experience, yet existing evaluations typically optimize under fixed execution conditions and do not test recovery after those conditions change. To address this gap, we introduce PACE-Bench (Physics Adaptation via Code Evolution), a simulator-grounded benchmark of 144 source-to-target adaptation pairs across six physics domains. Each pair links a source environment to a mutated target environment with the same goal and interface. A code-driven design that succeeds in the source fails in the target, where agents must iteratively adapt it into a working target design using diagnostic sandbox feedback within a limited attempt budget. We compare ten self-evolving methods from four paradigms. The benchmark remains far from saturated: Reflexion + Qwen3-14B succeeds on only 35.9\% of full-benchmark pairs, while GPT-5.5 solves 66.7\% of the Statics subset under the full budget. Together, these results show that simulator-grounded reflection is more reliable than unverified self-revision, while memory anchors agents to early designs and broad tree search explores without converging. Even revealing exact physical changes does not raise the performance ceiling, pointing to mechanism redesign rather than parameter inference as the central bottleneck. Data and code are available at https://github.com/thunlp/PACE-Bench.

Related papers

• SPADE: Self-Play in Adaptive Synthetic Executable Environments [110.80471334001574]
  We introduce SPADE, a self-play RL framework in which a single LLM plays two roles.<n>The Environment Designer writes complete, long-horizon training environments as executable code with an OpenAI Gym-style reset()/step() interface.<n>We find several components critical to success: grounding the Environment Designer on documents sampled from a large pretraining corpus, and giving it an accumulated environment memory.
  arXiv  Detail & Related papers  (2026-08-19T17:58:56Z)
• Rethinking Self-Evolving Agent Skills: Feedback Dynamics over Multiple Rounds [43.5891705224868]
  Self-evolving skill systems promise to improve agents by turning execution feedback into persistent skill updates without changing the underlying model.<n>We present a controlled evaluation framework across five benchmarks and three models.<n>Overall, persistent skill self-evolution is better understood as sparse, validation-filtered search with model- and benchmark-dependent returns.
  arXiv  Detail & Related papers  (2026-07-31T04:02:51Z)
• Amplitude-Only FFN Intervention for Tool-Structured LLM Inference Method: Gated Evaluation Protocol, and Cross-Model Empirical Results [10.526208975530054]
  We study inference-time feed-forward network (FFN) intervention for improving structured outputs without retraining model weights.<n>Amplitude Gating (AG) is a non-destructive alternative that preserves pretrained FFN weight directions and modulates only activation magnitudes during generation.<n>These results identify tool-structured inference as the most credible first target for safe FFN-level inference optimization.
  arXiv  Detail & Related papers  (2026-07-13T07:28:55Z)
• MAGE: Understanding Stability-Performance Trade-offs in Multi-component Prompt Optimization [3.7323766833175718]
  MAGE (Memory-Augmented Goal Prompt Evolution) is a controlled analysis framework for studying component interaction in prompt optimization.<n>Our experiments uncover a previously unreported phenomenon, the Prompt Optimization Coupling Effect (POCE), when multiple optimization signals operate within a closed reflective loop.
  arXiv  Detail & Related papers  (2026-07-11T10:05:58Z)
• GRASP: Gated Regression-Aware Skill Proposer for Self-Improving LLM Agents [17.26555663132631]
  GRASP (Gated Regression-Aware Skill Proposer) treats agent improvement as a sequence of edits to a bounded skill library.<n>We evaluate GRASP across five base models on clinical benchmarks.
  arXiv  Detail & Related papers  (2026-05-28T09:30:09Z)
• Computer Use at the Edge of the Statistical Precipice [19.604476051265852]
  We show that a 1MB replay script that blindly executes a recorded action sequence outperforms frontier models on prominent static benchmarks.<n>We trace failures to two root causes: non-principled environment design and non-principled evaluation methodology.
  arXiv  Detail & Related papers  (2026-05-07T22:43:43Z)
• ClawArena: Benchmarking AI Agents in Evolving Information Environments [61.664633997138004]
  ClawArena is a benchmark for evaluating AI agents in evolving information environments.<n>Each scenario maintains a complete hidden ground truth while exposing the agent only to noisy, partial, and sometimes contradictory traces.<n> evaluation is organized around three coupled challenges: multi-source conflict reasoning, dynamic belief revision, and implicit personalization.
  arXiv  Detail & Related papers  (2026-04-05T17:55:23Z)
• Dynamic analysis enhances issue resolution [53.50448142467294]
  DAIRA (Dynamic Analysis-enhanced Issue Resolution Agent) is an automated repair framework that embeds dynamic analysis into the agent's reasoning cycle.<n> Driven by a Test Tracing-Driven methodology, DAIRA utilizes lightweight monitors to extract critical runtime data.<n>When powered by Gemini 3 Flash Preview, DAIRA establishes a new state-of-the-art (SOTA) performance, achieving a 79.4% resolution rate on the SWE-bench Verified dataset.
  arXiv  Detail & Related papers  (2026-03-23T14:48:54Z)
• CORE: Context-Robust Remasking for Diffusion Language Models [51.59514489363897]
  We propose Context-Robust Remasking (CORE), a training-free framework for inference-time revision.<n>Rather than trusting static token probabilities, CORE identifies context-brittle tokens by probing their sensitivity to targeted masked-context perturbations.<n>On LLaDA-8B-Base, CORE delivers consistent improvements across reasoning and code benchmarks, outperforming compute-matched baselines and improving MBPP by up to 9.2 percentage points.
  arXiv  Detail & Related papers  (2026-02-04T00:12:30Z)
• Continual Adaptation: Environment-Conditional Parameter Generation for Object Detection in Dynamic Scenarios [54.58186816693791]
  environments constantly change over time and space, posing significant challenges for object detectors trained based on a closed-set assumption.<n>We propose a new mechanism, converting the fine-tuning process to a specific- parameter generation.<n>In particular, we first design a dual-path LoRA-based domain-aware adapter that disentangles features into domain-invariant and domain-specific components.
  arXiv  Detail & Related papers  (2025-06-30T17:14:12Z)
• Evolving Pareto-Optimal Actor-Critic Algorithms for Generalizability and Stability [67.8426046908398]
  Generalizability and stability are two key objectives for operating reinforcement learning (RL) agents in the real world. This paper presents MetaPG, an evolutionary method for automated design of actor-critic loss functions.
  arXiv  Detail & Related papers  (2022-04-08T20:46:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.