Vorch-Director: Interactive World Story Model via Noise-Aware Error Rectification

• URL: http://arxiv.org/abs/2608.05776v1
• Date: Thu, 06 Aug 2026 09:09:49 GMT
• Title: Vorch-Director: Interactive World Story Model via Noise-Aware Error Rectification
• Abstract summary: Vorch-Director is a noise-level-aware residual correction strategy.<n>It associates each residual with its originating noise level and injects residuals from matched noise regimes during training.<n>It produces more realistic autoregressive histories while retaining efficient teacher-forcing training.
• Score: 28.964205712747255
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Autoregressive continuation provides a natural path toward minute-scale audio-visual generation by repeatedly extending a short-window generator conditioned on previously generated video and audio. However, models are trained on clean ground-truth histories, while inference relies on their own generated histories, where accumulated errors cause identity drift, over-smoothing, and audio-visual desynchronization. Recent methods reduce this mismatch by reusing prediction residuals as synthetic corruption, but we observe that the effectiveness of residual correction critically depends on the flow-matching noise level at which residuals are produced. We propose Vorch-Director, a noise-level-aware residual correction strategy that associates each residual with its originating noise level and injects residuals from matched noise regimes during training. By aligning injected errors with the denoising process, Vorch-Director produces more realistic autoregressive histories while retaining efficient teacher-forcing training. Built on the audio-visual LTX-2 diffusion transformer, Vorch-Director further introduces task embeddings to distinguish historical video, reference images, and target video, enabling unified conditioning for long-horizon generation. Together with a clean conditioning sink and mixed-task training, Vorch-Director supports multi-shot, multi-subject, reference-guided audio-visual long-video generation. We evaluate Vorch-Director on ST-Bench and introduce a new long-horizon audio-visual benchmark with metrics for quality drift and long-range consistency. Extensive experiments demonstrate improved stability and audio-visual fidelity over strong baselines.

Related papers

• Equilibrium Forcing: Adaptive Video Generation Without Noise Conditioning [85.93126001283169]
  We introduce Equilibrium Forcing (EqF), a framework for video denoising generative models without noise level conditioning.<n>This flexibility allows for inference-time algorithms that operate in a closed loop by adapting to feedback from the sample.
  arXiv  Detail & Related papers  (2026-08-11T00:02:32Z)
• Test-Time Self-Adaptive Conditioning for Stable Audio-Driven Talking-Head Generation [28.732587811107777]
  We introduce Test-Time Self-Adaptive Conditioning (TT-SAC), a parameter-free inference framework that enables pretrained talking-head generators to adapt their conditioning representations during inference without retraining, gradient updates, or additional supervision.<n>A single adaptation step approximates a self-consistent equilibrium of the generative process, stabilizing identity and motion across time.<n>Experiments on state-of-the-art talking-head generators and benchmark datasets demonstrate consistent improvements in lip-sync accuracy, temporal coherence, identity preservation, and perceptual fidelity.
  arXiv  Detail & Related papers  (2026-05-25T06:45:29Z)
• RAVEN: Real-time Autoregressive Video Extrapolation with Consistency-model GRPO [53.38929612273108]
  Causal autoregressive video diffusion models support real-time streaming generation by extrapolating future chunks from previously generated content.<n>We introduce the Real-time Autoregressive Video Extrapolation Network (RAVEN), a training-time test framework that repacks each self rollout into an interleaved sequence of clean historical endpoints.
  arXiv  Detail & Related papers  (2026-05-14T17:59:30Z)
• Accelerating Training of Autoregressive Video Generation Models via Local Optimization with Representation Continuity [57.83511884904928]
  We explore methods to accelerate training for autoregressive video generation models through empirical analyses.<n>Our results reveal that while training on fewer video frames significantly reduces training time, it also exacerbates error accumulation and introduces inconsistencies in the generated videos.<n>Inspired by Lipschitz continuity, we propose a Representation Continuity (ReCo) strategy to improve the consistency of generated videos.
  arXiv  Detail & Related papers  (2026-04-08T09:43:03Z)
• Conditional Flow Matching for Visually-Guided Acoustic Highlighting [10.241097929338366]
  Visually-guided acoustic highlighting seeks to rebalance audio in alignment with the accompanying video, creating a coherent audio-visual experience.<n>Existing approaches use discriminative models, which struggle with the inherent ambiguity in audio remixing.<n>We introduce a Conditional Flow Matching (CFM) framework to address this limitation.
  arXiv  Detail & Related papers  (2026-02-03T17:24:47Z)
• End-to-End Training for Autoregressive Video Diffusion via Self-Resampling [63.84672807009907]
  Autoregressive video diffusion models hold promise for world simulation but are vulnerable to exposure bias arising from the train-test mismatch.<n>We introduce Resampling Forcing, a teacher-free framework that enables training autoregressive video models from scratch and at scale.
  arXiv  Detail & Related papers  (2025-12-17T18:53:29Z)
• Adaptive Begin-of-Video Tokens for Autoregressive Video Diffusion Models [11.913945404405865]
  Most video diffusion models (VDMs) generate videos in an autoregressive manner, generating subsequent iteration frames conditioned on previous ones.<n>We propose Adaptive Begin-of-Video Tokens (ada-BOV) for autoregressive VDMs.
  arXiv  Detail & Related papers  (2025-11-15T08:29:14Z)
• Stable Video Infinity: Infinite-Length Video Generation with Error Recycling [76.91310169118408]
  We propose Stable Video Infinity (SVI) that is able to generate infinite-length videos with high temporal consistency, plausible scene transitions, and controllable streaming storylines.<n> SVI incorporates Error-Recycling Fine-Tuning, a new type of efficient training that recycles the Diffusion Transformer's self-generated errors into supervisory prompts.<n>We evaluate SVI on three benchmarks, including consistent, creative, and conditional settings, thoroughly verifying its versatility and state-of-the-art role.
  arXiv  Detail & Related papers  (2025-10-10T09:45:46Z)
• Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion [67.94300151774085]
  We introduce Self Forcing, a novel training paradigm for autoregressive video diffusion models.<n>It addresses the longstanding issue of exposure bias, where models trained on ground-truth context must generate sequences conditioned on their own imperfect outputs.
  arXiv  Detail & Related papers  (2025-06-09T17:59:55Z)
• ScalingNoise: Scaling Inference-Time Search for Generating Infinite Videos [41.45750971432533]
  Video diffusion models (VDMs) facilitate the generation of high-quality videos.<n>Recent studies have uncovered the existence of "golden noises" that can enhance video quality during generation.<n>We propose ScalingNoise, a plug-and-play inference-time search strategy that identifies golden initial noises for the diffusion sampling process.
  arXiv  Detail & Related papers  (2025-03-20T17:54:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.