Early Failure Detection and Intervention in Video Diffusion Models

• URL: http://arxiv.org/abs/2603.14320v1
• Date: Sun, 15 Mar 2026 10:32:41 GMT
• Title: Early Failure Detection and Intervention in Video Diffusion Models
• Abstract summary: We propose an early failure detection and diagnostic intervention pipeline for latent T2V diffusion models.<n>For detection, we design a Real-time Inspection (RI) module that converts latents into intermediate video previews.<n>Experiments on CogVideoX-5B and Wan2.1-1.3B demonstrate consistency gains on VBench with up to 2.64 times less time overhead.
• Score: 26.52125777621677
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text-to-video (T2V) diffusion models have rapidly advanced, yet generations still occasionally fail in practice, such as low text-video alignment or low perceptual quality. Since diffusion sampling is non-deterministic, it is difficult to know during inference whether a generation will succeed or fail, incurring high computational cost due to trial-and-error regeneration. To address this, we propose an early failure detection and diagnostic intervention pipeline for latent T2V diffusion models. For detection, we design a Real-time Inspection (RI) module that converts latents into intermediate video previews, enabling the use of established text-video alignment scorers for inspection in the RGB space. The RI module completes the conversion and inspection process in just 39.2ms. This is highly efficient considering that CogVideoX-5B requires 4.3s per denoising step when generating a 480p, 49-frame video on an NVIDIA A100 GPU. Subsequently, we trigger a hierarchical and early-exit intervention pipeline only when failure is predicted. Experiments on CogVideoX-5B and Wan2.1-1.3B demonstrate consistency gains on VBench with up to 2.64 times less time overhead compared to post-hoc regeneration. Our method also generalizes to a higher-capacity setting, remaining effective on Wan2.1-14B with 720p resolution and 81-frame generation. Furthermore, our pipeline is plug-and-play and orthogonal to existing techniques, showing seamless compatibility with prompt refinement and sampling guidance methods. We also provide evidence that failure signals emerge early in the denoising process and are detectable within intermediate video previews using standard vision-language evaluators.

Related papers

• Strictly Causal Streaming Video Anomaly Detection with a Theoretically-Grounded State-Space Core [2.648500779572419]
  We introduce a causal streaming anomaly detector with no lookahead and no clip buffering.<n>Its temporal core is a diagonal linear state space recurrence with an input and state dependent decay gate.<n>We derive a closed form relationship between the recurrence decay spectrum and both detection delay and the shortest anomaly it can reliably capture.
  arXiv  Detail & Related papers  (2026-08-25T16:52:32Z)
• Video-DeepResearch: Towards the Next-Generation Multimodal Deepresearch Agent [81.19460113054048]
  Video-DR features a decoupled perception-temporal pipeline with stage-wise tool unlocking.<n>Our framework adopts a two-stage training recipe: supervised fine-tuning followed by Group Relative Policy Optimization.<n> Empirical results demonstrate that our Video-ResearchDeepB-A3B establishes a new state-of-the-art of 64.0% average accuracy.
  arXiv  Detail & Related papers  (2026-08-04T17:45:16Z)
• DiffusionGemma Technical Report [72.10570774996219]
  We introduce DiffusionGemma, a language model that uses discrete diffusion to generate text at exceptionally high speed.<n>Rather than decoding one token at a time, DiffusionGemma iteratively refines blocks of 256 tokens in parallel.<n>Our compute-efficient two-stage training pipeline uses fewer than 10% of the starting AR model's total training token budget.
  arXiv  Detail & Related papers  (2026-07-31T16:11:46Z)
• DTG-Restore: Training-Free Diffusion Refinement for Generative Video Super-Resolution [11.720438605063748]
  We introduce a training-free framework that enhances distorted and low-resolution videos by decoupling conditional and unconditional branches in time.<n>Our proposed Decoupled Time Guidance (DTG) evaluates the unconditional branch at a cleaner diffusion timestep.<n>To facilitate evaluation, we curate GenWarp480, a benchmark of 4,400 distorted 480p videos synthesized from diverse text-to-video models.
  arXiv  Detail & Related papers  (2026-05-28T18:00:14Z)
• Streaming Autoregressive Video Generation via Diagonal Distillation [50.13573884115673]
  Autoregressive models offer a natural framework for sequential frame synthesis but require heavy computation to achieve high fidelity.<n>We propose Diagonal Distillation to exploit temporal information across both video chunks and denoising steps.<n>Our method generates a 5-second video in 2.61 seconds (up to 31 FPS), achieving a 277.3x speedup over the undistilled model.
  arXiv  Detail & Related papers  (2026-03-10T10:45:24Z)
• Adaptive Spectral Feature Forecasting for Diffusion Sampling Acceleration [58.19554276924402]
  We propose spectral diffusion feature forecaster (Spectrum) to enable global, long-range feature reuse with tightly controlled error.<n>We achieve up to 4.79$times$ speedup on FLUX.1 and 4.67$times$ speedup on Wan2.1-14B, while maintaining much higher sample quality compared with the baselines.
  arXiv  Detail & Related papers  (2026-03-02T08:59:11Z)
• Pseudo Anomalies Are All You Need: Diffusion-Based Generation for Weakly-Supervised Video Anomaly Detection [2.4832691511446163]
  We introduce PA-VAD, a generation-driven approach that learns a detector from synthesized pseudo-abnormal videos paired with real normal videos.<n>For synthesis, we select class-relevant initial images with CLIP and refine prompts with a vision-language model to improve fidelity and scene consistency.<n>For training, we mitigate excessive video magnitude in synthesized anomalies by an domain-aligned regularized module.
  arXiv  Detail & Related papers  (2025-12-07T13:43:19Z)
• BLADE: Block-Sparse Attention Meets Step Distillation for Efficient Video Generation [27.57431718095974]
  We present BLADE, a data-free joint training framework for video inference.<n>We show remarkable efficiency gains across different scales.<n>On models such as CogVideoX-5B with short video sequence lengths, our framework delivers a robust 8.89x speedup.
  arXiv  Detail & Related papers  (2025-08-14T15:58:59Z)
• From Slow Bidirectional to Fast Autoregressive Video Diffusion Models [48.35054927704544]
  Current video diffusion models achieve impressive generation quality but struggle in interactive applications due to bidirectional attention dependencies.<n>We address this limitation by adapting a pretrained bidirectional diffusion transformer to an autoregressive transformer that generates frames on-the-fly.<n>Our model achieves a total score of 84.27 on the VBench-Long benchmark, surpassing all previous video generation models.
  arXiv  Detail & Related papers  (2024-12-10T18:59:50Z)
• StreamDiffusion: A Pipeline-level Solution for Real-time Interactive Generation [52.56469577812338]
  We introduce StreamDiffusion, a real-time diffusion pipeline for interactive image generation.<n>Existing diffusion models are adept at creating images from text or image prompts, yet they often fall short in real-time interaction.<n>We present a novel approach that transforms the original sequential denoising into the denoising process.
  arXiv  Detail & Related papers  (2023-12-19T18:18:33Z)
• DiffusionVMR: Diffusion Model for Joint Video Moment Retrieval and Highlight Detection [38.12212015133935]
  A novel framework, DiffusionVMR, is proposed to redefine the two tasks as a unified conditional denoising generation process. Experiments conducted on five widely-used benchmarks demonstrate the effectiveness and flexibility of the proposed DiffusionVMR.
  arXiv  Detail & Related papers  (2023-08-29T08:20:23Z)
• Robust Unsupervised Video Anomaly Detection by Multi-Path Frame Prediction [61.17654438176999]
  We propose a novel and robust unsupervised video anomaly detection method by frame prediction with proper design. Our proposed method obtains the frame-level AUROC score of 88.3% on the CUHK Avenue dataset.
  arXiv  Detail & Related papers  (2020-11-05T11:34:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.