Exposure Bias as Epistemic Underidentification in Recursive Forecasting

• URL: http://arxiv.org/abs/2606.12990v1
• Date: Thu, 11 Jun 2026 07:25:15 GMT
• Title: Exposure Bias as Epistemic Underidentification in Recursive Forecasting
• Abstract summary: We show that, under partial observability or state truncation, recursive rollout is also an underidentification problem.<n>Even with deterministic latent dynamics, one-step Bayes supervision identifies behavior only on observed contexts.<n>We formalize this with induced states $Z$ and provenance variables $P$, and derive a decomposition of induced-state error into teacher-forcing/rollout mismatch.
• Score: 1.2489632787815885
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recursive multi-step forecasting is usually framed as distribution shift: models are trained on observed histories but deployed on their own predictions. We show this framing is incomplete by proving that, under partial observability or state truncation, recursive rollout is also an epistemic underidentification problem. Even with deterministic latent dynamics, one-step Bayes supervision identifies behavior only on observed contexts and need not identify the deployed recursive predictor once rollout queries self-generated induced states whose correct local targets are not determined by numeric state alone. We formalize this with induced states $Z$ and provenance variables $P$, and derive a decomposition of induced-state error into teacher-forcing/rollout mismatch, representation--class approximation, and provenance information gaps. Empirically, we show that rollout enters a distinct induced-state regime, that fixed induced states define a distinct local corrective task, and that closed-loop gains arise not only from local adaptation but also from changing the induced states visited during rollout. Using a simple binary provenance encoding, provenance-aware correction can further improve performance, though gains are conditional rather than uniform. These results recast exposure bias as reasoning under self-induced epistemic uncertainty.

Related papers

• Replicating Belief, Not Bits: Epistemic State Replication for Agentic Systems [2.124730017640531]
  In distributed systems, the classical State Machine Replication model assumes that correct replicas execute deterministic transitions to yield identical bitwise states.<n>We propose Epistemic State Replication (ESR), a belief-replication layer for agentic distributed systems that shifts the replication boundary from data visibility to knowledge visibility.
  arXiv  Detail & Related papers  (2026-07-03T19:59:03Z)
• The Need for an External Observer Formalizing the Sufficiency Gap: A Mathematical Extension of Mixture Identifiability and Contextual Grounding in Sequence Models [0.0]
  We construct a binary mixed-regime process with one deterministic textual regime and one random regime governed by an unobserved latent state.<n>The resulting entropy difference is not an ordinary optimization error; it is a sufficiency gap caused by marginalization over an unobserved state.<n>A corrective signal reverses the posterior odds induced by the textual history exactly when its fidelity exceeds the text-only posterior weight assigned to the misleading regime.
  arXiv  Detail & Related papers  (2026-05-26T08:53:11Z)
• Dynamic Correction of Erroneous State Estimates via Diffusion Bayesian Exploration [9.087312455936134]
  In emergency response and other high-stakes societal applications, early-stage state estimates critically shape downstream outcomes.<n>These initial state estimates-often based on limited or biased information-can be severely misaligned with reality.<n>We propose a diffusion-driven Bayesian exploration framework that enables principled, real-time correction of early state estimation errors.
  arXiv  Detail & Related papers  (2025-12-01T22:08:26Z)
• Selecting Belief-State Approximations in Simulators with Latent States [6.368520403252146]
  State resetting is a fundamental but often overlooked capability of simulators.<n>We show that this problem reduces to a general conditional distribution-selection task.<n>We develop a new algorithm and analysis under sampling-only access.
  arXiv  Detail & Related papers  (2025-11-25T21:34:01Z)
• Revisiting Multivariate Time Series Forecasting with Missing Values [65.30332997607141]
  Missing values are common in real-world time series.<n>Current approaches have developed an imputation-then-prediction framework that uses imputation modules to fill in missing values, followed by forecasting on the imputed data.<n>This framework overlooks a critical issue: there is no ground truth for the missing values, making the imputation process susceptible to errors that can degrade prediction accuracy.<n>We introduce Consistency-Regularized Information Bottleneck (CRIB), a novel framework built on the Information Bottleneck principle.
  arXiv  Detail & Related papers  (2025-09-27T20:57:48Z)
• Counterfactual Realizability [52.85109506684737]
  We introduce a formal definition of realizability, the ability to draw samples from a distribution, and then develop a complete algorithm to determine whether an arbitrary counterfactual distribution is realizable.<n>We illustrate the implications of this new framework for counterfactual data collection using motivating examples from causal fairness and causal reinforcement learning.
  arXiv  Detail & Related papers  (2025-03-14T20:54:27Z)
• Identifiable Latent Neural Causal Models [82.14087963690561]
  Causal representation learning seeks to uncover latent, high-level causal representations from low-level observed data. We determine the types of distribution shifts that do contribute to the identifiability of causal representations. We translate our findings into a practical algorithm, allowing for the acquisition of reliable latent causal representations.
  arXiv  Detail & Related papers  (2024-03-23T04:13:55Z)
• Temporally Disentangled Representation Learning under Unknown Nonstationarity [35.195001384417395]
  We introduce NCTRL, a principled estimation framework, to reconstruct time-delayed latent causal variables. Empirical evaluations demonstrated the reliable identification of time-delayed latent causal influences.
  arXiv  Detail & Related papers  (2023-10-28T06:46:03Z)
• Nonparametric Identifiability of Causal Representations from Unknown Interventions [63.1354734978244]
  We study causal representation learning, the task of inferring latent causal variables and their causal relations from mixtures of the variables. Our goal is to identify both the ground truth latents and their causal graph up to a set of ambiguities which we show to be irresolvable from interventional data.
  arXiv  Detail & Related papers  (2023-06-01T10:51:58Z)
• Neuro-Symbolic Entropy Regularization [78.16196949641079]
  In structured prediction, the goal is to jointly predict many output variables that together encode a structured object. One approach -- entropy regularization -- posits that decision boundaries should lie in low-probability regions. We propose a loss, neuro-symbolic entropy regularization, that encourages the model to confidently predict a valid object.
  arXiv  Detail & Related papers  (2022-01-25T06:23:10Z)
• Which Invariance Should We Transfer? A Causal Minimax Learning Approach [18.71316951734806]
  We present a comprehensive minimax analysis from a causal perspective. We propose an efficient algorithm to search for the subset with minimal worst-case risk. The effectiveness and efficiency of our methods are demonstrated on synthetic data and the diagnosis of Alzheimer's disease.
  arXiv  Detail & Related papers  (2021-07-05T09:07:29Z)
• The Hidden Uncertainty in a Neural Networks Activations [105.4223982696279]
  The distribution of a neural network's latent representations has been successfully used to detect out-of-distribution (OOD) data. This work investigates whether this distribution correlates with a model's epistemic uncertainty, thus indicating its ability to generalise to novel inputs.
  arXiv  Detail & Related papers  (2020-12-05T17:30:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.