Curiosity-Critic: Cumulative Prediction Error Improvement as a Tractable Intrinsic Reward for World Model Training

• URL: http://arxiv.org/abs/2604.18701v2
• Date: Wed, 29 Apr 2026 05:00:55 GMT
• Title: Curiosity-Critic: Cumulative Prediction Error Improvement as a Tractable Intrinsic Reward for World Model Training
• Abstract summary: Local prediction-error-based curiosity rewards focus on the current transition without considering the world model's cumulative prediction error.<n>We introduce Curiosity-Critic, which grounds its reward in the improvement of this cumulative objective.<n> Experiments on a grid world show that Curiosity-Critic outperforms prediction-error, visitation-count, and Random Network Distillation methods in training speed and final world model accuracy.
• Score: 2.112639121757009
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Local prediction-error-based curiosity rewards focus on the current transition without considering the world model's cumulative prediction error across all visited transitions. We introduce Curiosity-Critic, which grounds its intrinsic reward in the improvement of this cumulative objective, and show that it admits a tractable per-step surrogate: the difference between the current prediction error and the asymptotic error baseline of the current state transition. We estimate this error baseline online with a learned critic co-trained alongside the world model; regressing a single scalar, the critic converges well before the world model saturates, redirecting exploration toward learnable transitions without oracle knowledge of the noise floor. The reward is higher for learnable transitions and collapses toward the error baseline for stochastic ones, effectively separating epistemic (reducible) from aleatoric (irreducible) prediction error online. Prior prediction-error curiosity formulations, from Schmidhuber (1991) to learned-feature-space variants, emerge as special cases corresponding to specific approximations of this error baseline. Experiments on a stochastic grid world show that Curiosity-Critic outperforms prediction-error, visitation-count, and Random Network Distillation methods in training speed and final world model accuracy.

Related papers

• Heteroscedastic Diffusion for Multi-Agent Trajectory Modeling [18.081497075992406]
  Multi-agent trajectory modeling traditionally focuses on forecasting, often neglecting more general tasks like trajectory completion.<n>We introduce U2Diffine, a unified diffusion model built to perform trajectory completion while simultaneously offering state-wise heteroscedastic uncertainty estimates.<n>Our method outperforms state-of-the-art solutions in both trajectory completion and forecasting across four challenging sports datasets.
  arXiv  Detail & Related papers  (2026-05-11T15:26:14Z)
• Rectify, Don't Regret: Avoiding Pitfalls of Differentiable Simulation in Trajectory Prediction [8.730908751307737]
  Current open-loop trajectory models struggle in real-world autonomous driving because minor initial deviations cascade into compounding errors.<n>We introduce a detached receding horizon rollout to explicitly sever the graph between simulation steps.<n>Our approach yields more robust recovery strategies, reducing target collisions by up to 33.24%.
  arXiv  Detail & Related papers  (2026-03-24T16:28:09Z)
• Mitigating Error Accumulation in Continuous Navigation via Memory-Augmented Kalman Filtering [20.380488628232186]
  Continuous navigation is critical for Unmanned Aerial Vehicle (UAV)<n>Existing Vision-Language Navigation (VLN) models follow the dead-reckoning, which iteratively updates its position for the next waypoint prediction, and subsequently construct the complete trajectory.<n>We propose a novel framework that decouples navigation into two complementary processes: a Prior Prediction, based on motion dynamics and a Likelihood Correction, from historical observation.
  arXiv  Detail & Related papers  (2026-01-30T05:03:08Z)
• Revisiting Multivariate Time Series Forecasting with Missing Values [65.30332997607141]
  Missing values are common in real-world time series.<n>Current approaches have developed an imputation-then-prediction framework that uses imputation modules to fill in missing values, followed by forecasting on the imputed data.<n>This framework overlooks a critical issue: there is no ground truth for the missing values, making the imputation process susceptible to errors that can degrade prediction accuracy.<n>We introduce Consistency-Regularized Information Bottleneck (CRIB), a novel framework built on the Information Bottleneck principle.
  arXiv  Detail & Related papers  (2025-09-27T20:57:48Z)
• Unified Uncertainty-Aware Diffusion for Multi-Agent Trajectory Modeling [13.993231805213354]
  We introduce U2Diff, a textbfunified diffusion model designed to handle trajectory completion.<n>We also incorporate a Rank Neural Network in post-processing to enable textbferror probability estimation for each generated mode.<n>Our method outperforms the state-of-the-art solutions in trajectory completion and forecasting across four challenging sports datasets.
  arXiv  Detail & Related papers  (2025-03-24T11:46:58Z)
• Prediction-Powered Causal Inferences [59.98498488132307]
  We focus on Prediction-Powered Causal Inferences (PPCI)<n>We first show that conditional calibration guarantees valid PPCI at population level.<n>We then introduce a sufficient representation constraint transferring validity across experiments.
  arXiv  Detail & Related papers  (2025-02-10T10:52:17Z)
• Learning When the Concept Shifts: Confounding, Invariance, and Dimension Reduction [5.38274042816001]
  In observational data, the distribution shift is often driven by unobserved confounding factors. This motivates us to study the domain adaptation problem with observational data. We show a model that uses the learned lower-dimensional subspace can incur nearly ideal gap between target and source risk.
  arXiv  Detail & Related papers  (2024-06-22T17:43:08Z)
• Selective Learning: Towards Robust Calibration with Dynamic Regularization [79.92633587914659]
  Miscalibration in deep learning refers to there is a discrepancy between the predicted confidence and performance. We introduce Dynamic Regularization (DReg) which aims to learn what should be learned during training thereby circumventing the confidence adjusting trade-off.
  arXiv  Detail & Related papers  (2024-02-13T11:25:20Z)
• EANet: Expert Attention Network for Online Trajectory Prediction [5.600280639034753]
  Expert Attention Network is a complete online learning framework for trajectory prediction. We introduce expert attention, which adjusts the weights of different depths of network layers, avoiding the model updated slowly due to gradient problem. Furthermore, we propose a short-term motion trend kernel function which is sensitive to scenario change, allowing the model to respond quickly.
  arXiv  Detail & Related papers  (2023-09-11T07:09:40Z)
• Learning Sample Difficulty from Pre-trained Models for Reliable Prediction [55.77136037458667]
  We propose to utilize large-scale pre-trained models to guide downstream model training with sample difficulty-aware entropy regularization. We simultaneously improve accuracy and uncertainty calibration across challenging benchmarks.
  arXiv  Detail & Related papers  (2023-04-20T07:29:23Z)
• Towards Out-of-Distribution Sequential Event Prediction: A Causal Treatment [72.50906475214457]
  The goal of sequential event prediction is to estimate the next event based on a sequence of historical events. In practice, the next-event prediction models are trained with sequential data collected at one time. We propose a framework with hierarchical branching structures for learning context-specific representations.
  arXiv  Detail & Related papers  (2022-10-24T07:54:13Z)
• Domain-Adjusted Regression or: ERM May Already Learn Features Sufficient for Out-of-Distribution Generalization [52.7137956951533]
  We argue that devising simpler methods for learning predictors on existing features is a promising direction for future research. We introduce Domain-Adjusted Regression (DARE), a convex objective for learning a linear predictor that is provably robust under a new model of distribution shift. Under a natural model, we prove that the DARE solution is the minimax-optimal predictor for a constrained set of test distributions.
  arXiv  Detail & Related papers  (2022-02-14T16:42:16Z)
• A heteroencoder architecture for prediction of failure locations in porous metals using variational inference [1.2722697496405462]
  We employ an encoder-decoder convolutional neural network to predict the failure locations of porous metal tension specimens. The objective of predicting failure locations presents an extreme case of class imbalance since most of the material in the specimens do not fail. We demonstrate that the resulting predicted variances are effective in ranking the locations that are most likely to fail in any given specimen.
  arXiv  Detail & Related papers  (2022-01-31T20:26:53Z)
• HYPER: Learned Hybrid Trajectory Prediction via Factored Inference and Adaptive Sampling [27.194900145235007]
  We introduce HYPER, a general and expressive hybrid prediction framework. By modeling traffic agents as a hybrid discrete-continuous system, our approach is capable of predicting discrete intent changes over time. We train and validate our model on the Argoverse dataset, and demonstrate its effectiveness through comprehensive ablation studies and comparisons with state-of-the-art models.
  arXiv  Detail & Related papers  (2021-10-05T20:20:10Z)
• Improving Uncertainty Calibration via Prior Augmented Data [56.88185136509654]
  Neural networks have proven successful at learning from complex data distributions by acting as universal function approximators. They are often overconfident in their predictions, which leads to inaccurate and miscalibrated probabilistic predictions. We propose a solution by seeking out regions of feature space where the model is unjustifiably overconfident, and conditionally raising the entropy of those predictions towards that of the prior distribution of the labels.
  arXiv  Detail & Related papers  (2021-02-22T07:02:37Z)
• Understanding and Mitigating the Tradeoff Between Robustness and Accuracy [88.51943635427709]
  Adversarial training augments the training set with perturbations to improve the robust error. We show that the standard error could increase even when the augmented perturbations have noiseless observations from the optimal linear predictor.
  arXiv  Detail & Related papers  (2020-02-25T08:03:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.