Rollout-Decoded Reconstruction for Long-Horizon Prediction in Latent World Models

• URL: http://arxiv.org/abs/2608.25017v1
• Date: Tue, 25 Aug 2026 18:05:24 GMT
• Title: Rollout-Decoded Reconstruction for Long-Horizon Prediction in Latent World Models
• Abstract summary: A latent world model trains its decoder on latents anchored to observations, then deploys it on the model's own free-running rollout, hundreds of steps past the last observation.<n>Rollout-Decoded Reconstruction (RDR) closes this gap with a single loss term that free-runs the model during training exactly as evaluation will, decodes every rollout latent, and penalizes reconstruction error against ground truth.
• Score: 0.9740025522928777
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A latent world model trains its decoder on latents anchored to observations, then deploys it on the model's own free-running rollout, hundreds of steps past the last observation. Rollout-Decoded Reconstruction (RDR) closes this gap with a single loss term that free-runs the model during training exactly as evaluation will, decodes every rollout latent, and penalizes reconstruction error against ground truth. The term adds no parameters, costs training-time compute only, and reduces to the standard objective at weight zero, so every comparison in this paper is a one-flag A/B. On the chaotic Kuramoto-Sivashinsky equation, RDR raises valid prediction time (the time to first crossing of normalized error 0.5) from $3.87 \pm 0.23$ to $6.97 \pm 0.42$ time units at an identical 193,568 parameters, a $1.80\times$ improvement confirmed on seeds never used in selection and holding in 10 of 10 preregistered configurations at ratios of 1.71-2.50$\times$. The results come from a single system; a sweep in which the advantage grows with latent width is descriptive, and control experiments on two classic tasks are preliminary.

Related papers

• Your VLM Already Knows When: Training-Free Temporal Grounding by Asking Yes or No [2.531680975633497]
  Multimodal LLMs that recognise events reliably still fail to say when they happen.<n>We show that this failure lives in the task interface, not in perception.
  arXiv  Detail & Related papers  (2026-08-08T19:57:47Z)
• Round-Trip Consistency: Bidirectional Diffusion Models Can Predict Their Own Rollout Errors [51.56484100374058]
  Autoregressive models accumulate error over long rollouts, yet at deployment there is no ground truth to measure it against.<n>We train a single conditional latent diffusion model that steps a dynamical system forward or backward in time via a direction flag.<n>We show that this bidirectionality supplies a measurement-free test-time error signal.
  arXiv  Detail & Related papers  (2026-08-01T13:49:46Z)
• Data Driven Block Replacement Scheduling [0.0]
  We develop data-driven algorithms for maintaining $N$ independent identical machines under a textitblock replacement policy<n>The goal is to learn the cost-minimizing interval $k*$ from operational data when the lifetime distribution is unknown.<n>A complementary Kaplan--Meier renewal algorithm estimates the lifetime distribution noncensorally from censored data, achieving almost-sure policy consistency and empirically near-zero incremental regret at long horizons.
  arXiv  Detail & Related papers  (2026-07-16T17:31:15Z)
• ShortOPD: Recovering Pruned LLMs with Short-to-Long On-Policy Distillation [72.54604107217669]
  textscpass@$k$ nearly vanishes after compression, yet textscpass@$k$ recovers substantially under repeated sampling.<n>Recovery should train on the compressed model's own on-policy states with dense token-level supervision.<n>We propose textbfshortopd, a short-to-long OPD schedule that detects teacher-confirmed repetitive suffixes.
  arXiv  Detail & Related papers  (2026-07-14T17:50:50Z)
• ChainzRule: Sample-Efficient, Robust Deep Learning Across Tabular, NLP, and Vision Tasks [0.0]
  Production deep learning systems across enterprise domains operate under constraints that academic benchmarks routinely obscure.<n>We present ChainzRule (CR), a neural architecture replacing typical activations with learnable layers governed by Differential Regularization (DREG)<n>CR achieves $85.71% pm 2.01%$ on Pima Diabetes, $46.20% pm 0.37%$ on SST-5 sentiment classification with a frozen encoder, and $70.17%$ on Full ordinal regression with 3.2M parameters.
  arXiv  Detail & Related papers  (2026-05-23T01:52:50Z)
• Unextractable Protocol Models: Collaborative Training and Inference without Weight Materialization [58.14514930760722]
  We consider a decentralized setup in which the participants collaboratively train and serve a large neural network.<n>In this setup, we explore the possibility of unmaterializable weights, where a full weight set is never available to any one participant.<n>We introduce Unextractable Protocol Models (UPMs): a training and inference framework that leverages the sharded model setup.
  arXiv  Detail & Related papers  (2026-05-22T10:24:57Z)
• Representation Without Reward: A JEPA Audit for LLM Fine-Tuning [1.2691047660244335]
  Joint-embedding predictive architectures (JEPAs) propose that a model should learn more useful abstractions when trained to predict latent representations rather than observed outputs.<n>For autoregressive language-model fine-tuning the principle entails a stricter requirement: the induced hidden-state geometry must reach the language-model head emphand improve the decoded task metric.<n>We test that requirement under a fixed Llama-3.2-1B-Instruct LoRA harness on natural-language-to-regex generation.
  arXiv  Detail & Related papers  (2026-05-14T20:27:32Z)
• MARBLE: Multi-Aspect Reward Balance for Diffusion RL [71.6241143519038]
  Reinforcement learning has become the dominant approach for aligning diffusion models with human preferences.<n>Existing practice deal with multiple rewards by training one specialist model per reward.<n>We propose MARBLE, a gradient-space optimization framework that maintains independent advantage estimators for each reward.
  arXiv  Detail & Related papers  (2026-05-07T16:20:42Z)
• SOAR: Self-Correction for Optimal Alignment and Refinement in Diffusion Models [48.335262141752715]
  Post-training pipeline for diffusion models has two stages: supervised fine-tuning (SFT) on curated data and reinforcement learning (RL) with reward models.<n>We propose SOAR (Self-Correction for Optimal Alignment and Refinement), a bias-correction post-training method that fills this gap.<n>It is on-policy, reward-free, and provides dense per-time supervision with no credit-assignment problem.
  arXiv  Detail & Related papers  (2026-04-14T11:45:15Z)
• ZIP-RC: Optimizing Test-Time Compute via Zero-Overhead Joint Reward-Cost Prediction [57.799425838564]
  We present ZIP-RC, an adaptive inference method that equips models with zero-overhead inference-time predictions of reward and cost.<n> ZIP-RC improves accuracy by up to 12% over majority voting at equal or lower average cost.
  arXiv  Detail & Related papers  (2025-12-01T09:44:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.