Selecting Belief-State Approximations in Simulators with Latent States

• URL: http://arxiv.org/abs/2511.20870v1
• Date: Tue, 25 Nov 2025 21:34:01 GMT
• Title: Selecting Belief-State Approximations in Simulators with Latent States
• Abstract summary: State resetting is a fundamental but often overlooked capability of simulators.<n>We show that this problem reduces to a general conditional distribution-selection task.<n>We develop a new algorithm and analysis under sampling-only access.
• Score: 6.368520403252146
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: State resetting is a fundamental but often overlooked capability of simulators. It supports sample-based planning by allowing resets to previously encountered simulation states, and enables calibration of simulators using real data by resetting to states observed in real-system traces. While often taken for granted, state resetting in complex simulators can be nontrivial: when the simulator comes with latent variables (states), state resetting requires sampling from the posterior over the latent state given the observable history, a.k.a. the belief state (Silver and Veness, 2010). While exact sampling is often infeasible, many approximate belief-state samplers can be constructed, raising the question of how to select among them using only sampling access to the simulator. In this paper, we show that this problem reduces to a general conditional distribution-selection task and develop a new algorithm and analysis under sampling-only access. Building on this reduction, the belief-state selection problem admits two different formulations: latent state-based selection, which directly targets the conditional distribution of the latent state, and observation-based selection, which targets the induced distribution over the observation. Interestingly, these formulations differ in how their guarantees interact with the downstream roll-out methods: perhaps surprisingly, observation-based selection may fail under the most natural roll-out method (which we call Single-Reset) but enjoys guarantees under the less conventional alternative (which we call Repeated-Reset). Together with discussion on issues such as distribution shift and the choice of sampling policies, our paper reveals a rich landscape of algorithmic choices, theoretical nuances, and open questions, in this seemingly simple problem.

Related papers

• Self-Balancing Sequential Sampling: Fast Convergence with Controlled Predictability [0.7734726150561088]
  sequential sampling rules that adaptively bias sampling probabilities in order to achieve faster convergence of the empirical distribution to a desired target law.<n>Self-balancing sampler is simple to implement, arises naturally among a class of Markovian samplers.<n>Results support a practical framework for reducing repeated selections and long gaps in coverage without making future selections overly predictable.
  arXiv  Detail & Related papers  (2026-07-23T01:10:42Z)
• Learning with Simulators: No Regret in a Computationally Bounded World [65.78002253798827]
  We introduce the framework of simulatable processes, where the learner has access to a simulator that approximates the distribution generating the data.<n>We show that we can recover the same learning guarantees as in the classical setting with independent data, namely, error bounds that depend on the VC.
  arXiv  Detail & Related papers  (2026-06-11T17:02:24Z)
• Exposure Bias as Epistemic Underidentification in Recursive Forecasting [1.2489632787815885]
  We show that, under partial observability or state truncation, recursive rollout is also an underidentification problem.<n>Even with deterministic latent dynamics, one-step Bayes supervision identifies behavior only on observed contexts.<n>We formalize this with induced states $Z$ and provenance variables $P$, and derive a decomposition of induced-state error into teacher-forcing/rollout mismatch.
  arXiv  Detail & Related papers  (2026-06-11T07:25:15Z)
• Overcoming Selection Bias in Statistical Studies With Amortized Bayesian Inference [1.9709458230153813]
  Selection bias arises when the probability that an observation enters a dataset depends on variables related to the quantities of interest.<n>We develop a bias-aware simulation-based inference framework that explicitly incorporates selection into neural posterior estimation.
  arXiv  Detail & Related papers  (2026-04-20T14:20:16Z)
• Simulation-Based Inference via Regression Projection and Batched Discrepancies [1.9435397960631862]
  We analyze a lightweight simulation-based inference method that infers simulator parameters using only a regression-based projection of the observed data.<n> Experiments on a tractable nonlinear model and on a cosmological calibration task using the DREAMS simulation suite illustrate the computational advantages of regression-based projections.
  arXiv  Detail & Related papers  (2026-02-03T15:07:40Z)
• Learning Shortest Paths When Data is Scarce [3.3012620893449465]
  We study a shortest-path problem in which a planner has access to abundant synthetic samples, limited real-world observations, and an edge-similarity capturing expected behavioral similarity across links.<n>We model the simulator-to-reality discrepancy as an unknown, edge-specific bias that varies smoothly over the similarity graph, and estimate it using Laplacian-regularized least squares.<n>For cold-start settings without initial real data, we develop a bias-aware active learning algorithm that adaptively selects edges to measure until a prescribed accuracy is met.
  arXiv  Detail & Related papers  (2026-01-07T06:19:04Z)
• Consistency of Selection Strategies for Fraud Detection [0.0]
  We study how insurers can chose which claims to investigate for fraud.<n>We argue that this can lead to inconsistent learning and propose a randomized alternative.
  arXiv  Detail & Related papers  (2025-09-23T07:33:33Z)
• Searching for a Hidden Markov Anomaly over Multiple Processes [13.582085518282849]
  We propose a novel algorithm, dubbed Anomaly Detection under Hidden Markov model (ADHM)<n>ADHM adapts the probing strategy based on accumulated statistical evidence and belief updates over hidden states.<n>It consistently outperforms existing methods in extensive simulations.
  arXiv  Detail & Related papers  (2025-06-20T16:10:38Z)
• Flipping Against All Odds: Reducing LLM Coin Flip Bias via Verbalized Rejection Sampling [59.133428586090226]
  Large language models (LLMs) can often accurately describe probability distributions using natural language.<n>This mismatch limits their use in tasks requiring reliableity, such as Monte Carlo methods, agent-based simulations, and randomized decision-making.<n>We introduce Verbalized Rejection Sampling (VRS), a natural-language adaptation of classical rejection sampling.
  arXiv  Detail & Related papers  (2025-06-11T17:59:58Z)
• Reliability analysis for non-deterministic limit-states using stochastic emulators [0.0]
  This paper introduces reliability analysis for models and addresses it by using suitable surrogate models to lower its typically high computational cost.<n>Specifically, we focus on the recently introduced generalized models and chaos expansions.<n>We validate our methodology through three case studies. First, using an analytical function with a closed-form solution, we demonstrate that the emulators converge to the correct solution.
  arXiv  Detail & Related papers  (2024-12-18T11:08:56Z)
• Conformal Language Modeling [61.94417935386489]
  We propose a novel approach to conformal prediction for generative language models (LMs) Standard conformal prediction produces prediction sets with rigorous, statistical guarantees. We demonstrate the promise of our approach on multiple tasks in open-domain question answering, text summarization, and radiology report generation.
  arXiv  Detail & Related papers  (2023-06-16T21:55:08Z)
• Sequential Predictive Two-Sample and Independence Testing [114.4130718687858]
  We study the problems of sequential nonparametric two-sample and independence testing. We build upon the principle of (nonparametric) testing by betting.
  arXiv  Detail & Related papers  (2023-04-29T01:30:33Z)
• Sample Efficient Deep Reinforcement Learning via Local Planning [21.420851589712626]
  This work focuses on sample-efficient deep reinforcement learning (RL) with a simulator. We propose an algorithmic framework, named uncertainty-first local planning (UFLP), that takes advantage of this property. We demonstrate that this simple procedure can dramatically improve the sample cost of several baseline RL algorithms on difficult exploration tasks.
  arXiv  Detail & Related papers  (2023-01-29T23:17:26Z)
• Sequential Kernelized Independence Testing [77.237958592189]
  We design sequential kernelized independence tests inspired by kernelized dependence measures.<n>We demonstrate the power of our approaches on both simulated and real data.
  arXiv  Detail & Related papers  (2022-12-14T18:08:42Z)
• Robust Continual Test-time Adaptation: Instance-aware BN and Prediction-balanced Memory [58.72445309519892]
  We present a new test-time adaptation scheme that is robust against non-i.i.d. test data streams. Our novelty is mainly two-fold: (a) Instance-Aware Batch Normalization (IABN) that corrects normalization for out-of-distribution samples, and (b) Prediction-balanced Reservoir Sampling (PBRS) that simulates i.i.d. data stream from non-i.i.d. stream in a class-balanced manner.
  arXiv  Detail & Related papers  (2022-08-10T03:05:46Z)
• Likelihood-Free Inference in State-Space Models with Unknown Dynamics [71.94716503075645]
  We introduce a method for inferring and predicting latent states in state-space models where observations can only be simulated, and transition dynamics are unknown. We propose a way of doing likelihood-free inference (LFI) of states and state prediction with a limited number of simulations.
  arXiv  Detail & Related papers  (2021-11-02T12:33:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.