Mind the Sim-to-Real Gap & Think Like a Scientist

• URL: http://arxiv.org/abs/2605.21458v1
• Date: Wed, 20 May 2026 17:48:14 GMT
• Title: Mind the Sim-to-Real Gap & Think Like a Scientist
• Abstract summary: We study when, and how, the planner should supplement the simulator with experiments.<n>We propose Fisher-SEP, a simulation-aided experimental policy.
• Score: 44.54570296032634
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Suppose a planner has a pre-trained simulator of a sequential decision problem and the option to run real experiments in the field. The simulator is cheap to query but inherits confounding and drift from its calibration data. Experimentation is unbiased but consumes one real unit per trial. We study when, and how, the planner should supplement the simulator with experiments. We give three results. First, an extended simulation lemma decomposes the simulator's value error into a calibration--deployment shift that randomization can identify and a parametric residual that no further interaction can reduce. Second, the value gap between the simulator-optimal policy and the optimum splits into a local component, on states the deployed policy already visits, and a reachability component, on states it does not. The reachability component stays bounded away from zero at any horizon under purely passive learning. Third, we propose Fisher-SEP, a simulation-aided experimental policy (SEP) that minimizes the posterior predictive variance of a target policy's value, with reward-only and transition-only specializations. Two case studies illustrate the regimes. In a vending-machine supply chain, front-loaded experimentation overtakes posterior updating once the horizon is long enough to amortize the pilot. In an HIV mobile-testing example with a corridor that separates a well-surveilled region from a poorly-surveilled one, only designed exploration reaches the poorly-surveilled region.

Related papers

• SCAPE: Scenario-Conditioned Simulation-Augmented Policy Evaluation [18.546902084606632]
  SCAPE is a scenario-conditioned simulation-augmented policy evaluation framework.<n>It predicts scenario-conditioned real-world policy performance using limited paired sim-and-real samples and large-scale simulation rollouts.<n>We validate SCAPE on autonomous driving and quadruped velocity tracking.
  arXiv  Detail & Related papers  (2026-08-19T20:24:29Z)
• One Frozen Simulator Is Not Enough: Simulator Collapse in Multi-Agent RL [75.34038228759182]
  Multi-agent reinforcement learning for human-AI interaction typically relies on a single large language model to simulate user behavior.<n>We show that this approach systematically fails to generalize, and trace the failure to simulator collapse.<n>We propose two complementary solutions, one at inference time and one at training time.
  arXiv  Detail & Related papers  (2026-08-12T16:55:50Z)
• Posterior-First Neural PDE Simulation: Inferring Hidden Problem State from a Single Field [5.24655241578805]
  A field-to-future predictor can collapse distinct latent problem states into the same deterministic interface.<n>We propose posterior-first neural PDE simulation: first infer a posterior over the minimal task-sufficient problem state, then condition prediction on that posterior.
  arXiv  Detail & Related papers  (2026-05-05T00:37:01Z)
• Learning Shortest Paths When Data is Scarce [3.3012620893449465]
  We study a shortest-path problem in which a planner has access to abundant synthetic samples, limited real-world observations, and an edge-similarity capturing expected behavioral similarity across links.<n>We model the simulator-to-reality discrepancy as an unknown, edge-specific bias that varies smoothly over the similarity graph, and estimate it using Laplacian-regularized least squares.<n>For cold-start settings without initial real data, we develop a bias-aware active learning algorithm that adaptively selects edges to measure until a prescribed accuracy is met.
  arXiv  Detail & Related papers  (2026-01-07T06:19:04Z)
• Selecting Belief-State Approximations in Simulators with Latent States [6.368520403252146]
  State resetting is a fundamental but often overlooked capability of simulators.<n>We show that this problem reduces to a general conditional distribution-selection task.<n>We develop a new algorithm and analysis under sampling-only access.
  arXiv  Detail & Related papers  (2025-11-25T21:34:01Z)
• PolySim: Bridging the Sim-to-Real Gap for Humanoid Control via Multi-Simulator Dynamics Randomization [53.7088694598817]
  We introduce PolySim, a WBC training platform that integrates multiple heterogeneous simulators.<n>Theoretically, we show that PolySim yields a tighter upper bound on simulator inductive bias than single-simulator training.
  arXiv  Detail & Related papers  (2025-10-02T06:31:42Z)
• Pseudo-Simulation for Autonomous Driving [66.1981253104508]
  Existing evaluation paradigms for Autonomous Vehicles (AVs) face critical limitations.<n>Real-world evaluation is often challenging due to safety concerns and a lack of realism.<n>Open-loop evaluation relies on metrics that generally overlook compounding errors.
  arXiv  Detail & Related papers  (2025-06-04T17:57:53Z)
• Prediction-Powered Causal Inferences [59.98498488132307]
  We focus on Prediction-Powered Causal Inferences (PPCI)<n>We first show that conditional calibration guarantees valid PPCI at population level.<n>We then introduce a sufficient representation constraint transferring validity across experiments.
  arXiv  Detail & Related papers  (2025-02-10T10:52:17Z)
• NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking [65.24988062003096]
  We present NAVSIM, a framework for benchmarking vision-based driving policies. Our simulation is non-reactive, i.e., the evaluated policy and environment do not influence each other. NAVSIM enabled a new competition held at CVPR 2024, where 143 teams submitted 463 entries, resulting in several new insights.
  arXiv  Detail & Related papers  (2024-06-21T17:59:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.