Latent-Compressed Variational Autoencoder for Video Diffusion Models

• URL: http://arxiv.org/abs/2604.16479v1
• Date: Sun, 12 Apr 2026 04:45:51 GMT
• Title: Latent-Compressed Variational Autoencoder for Video Diffusion Models
• Abstract summary: We propose a latent compression method that removes high-frequency components in video latent representations rather than directly reducing the number of channels.<n> Experimental results demonstrate that the proposed method achieves superior video reconstruction quality compared to strong baselines.
• Score: 27.770924931403997
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video variational autoencoders (VAEs) used in latent diffusion models typically require a sufficiently large number of latent channels to ensure high-quality video reconstruction. However, recent studies have revealed that an excessive number of latent channels can impede the convergence of latent diffusion models and deteriorate their generative performance, even when reconstruction quality remains high. We propose a latent compression method that removes high-frequency components in video latent representations rather than directly reducing the number of channels, which often compromises reconstruction fidelity. Experimental results demonstrate that the proposed method achieves superior video reconstruction quality compared to strong baselines while maintaining the same overall compression ratio.

Related papers

• LCUDiff: Latent Capacity Upgrade Diffusion for Faithful Human Body Restoration [23.264518366939825]
  Existing methods for restoring degraded human-centric images often struggle with insufficient fidelity.<n>We propose LCUDiff, a stable one-step framework that upgrades a pre-trained latent diffusion model.<n> Experiments on synthetic and real-world datasets show competitive results with higher fidelity and fewer artifacts.
  arXiv  Detail & Related papers  (2026-02-04T10:37:46Z)
• MTC-VAE: Multi-Level Temporal Compression with Content Awareness [54.85288415164888]
  Latent Video Diffusion Models (LVDMs) rely on Variational Autoencoders (VAEs) to compress videos into compact latent representations.<n>We present a technique to convert fixed compression rate VAEs into models that support multi-level temporal compression.
  arXiv  Detail & Related papers  (2026-02-01T17:08:02Z)
• LVTINO: LAtent Video consisTency INverse sOlver for High Definition Video Restoration [3.2944592608677614]
  We propose LVTINO, the first zero-shot or plug-and-play inverse solver for high definition video restoration with priors encoded by VCMs.<n>Our conditioning mechanism bypasses the need for automatic differentiation and achieves state-of-the-art video reconstruction quality with only a few neural function evaluations.
  arXiv  Detail & Related papers  (2025-10-01T18:10:08Z)
• Nuclear Diffusion Models for Low-Rank Background Suppression in Videos [20.045809197071204]
  Nuclear Diffusion is evaluated on a real-world medical imaging problem, namely cardiac ultrasound dehazing.<n>Results highlight the potential of combining model-based temporal models with deep generative priors for high-fidelity video restoration.
  arXiv  Detail & Related papers  (2025-09-25T08:20:22Z)
• DiTVR: Zero-Shot Diffusion Transformer for Video Restoration [48.97196894658511]
  DiTVR is a zero shot video restoration framework that couples a diffusion transformer with trajectory aware attention and a flow consistent sampler.<n>Our attention mechanism aligns tokens along optical flow trajectories, with particular emphasis on vital layers that exhibit the highest sensitivity to temporal dynamics.<n>The flow guided sampler injects data consistency only into low-frequency bands, preserving high frequency priors while accelerating cache.
  arXiv  Detail & Related papers  (2025-08-11T09:54:45Z)
• OSCAR: One-Step Diffusion Codec Across Multiple Bit-rates [39.746866725267516]
  Pretrained latent diffusion models have shown strong potential for lossy image compression.<n>We propose a one-step diffusion across multiple bit-rates termed OSCAR.<n>Experiments demonstrate that OSCAR achieves superior performance in both quantitative and visual quality metrics.
  arXiv  Detail & Related papers  (2025-05-22T00:14:12Z)
• Higher fidelity perceptual image and video compression with a latent conditioned residual denoising diffusion model [55.2480439325792]
  We propose a hybrid compression scheme optimized for perceptual quality, extending the approach of the CDC model with a decoder network.<n>We achieve up to +2dB PSNR fidelity improvements while maintaining comparable LPIPS and FID perceptual scores when compared with CDC.
  arXiv  Detail & Related papers  (2025-05-19T14:13:14Z)
• REGEN: Learning Compact Video Embedding with (Re-)Generative Decoder [52.698595889988766]
  We present a novel perspective on learning video embedders for generative modeling.<n>Rather than requiring an exact reproduction of an input video, an effective embedder should focus on visually plausible reconstructions.<n>We propose replacing the conventional encoder-decoder video embedder with an encoder-generator framework.
  arXiv  Detail & Related papers  (2025-03-11T17:51:07Z)
• Rethinking Video Tokenization: A Conditioned Diffusion-based Approach [58.164354605550194]
  New tokenizer, Diffusion Conditioned-based Gene Tokenizer, replaces GAN-based decoder with conditional diffusion model.<n>We trained using only a basic MSE diffusion loss for reconstruction, along with KL term and LPIPS perceptual loss from scratch.<n>Even a scaled-down version of CDT (3$times inference speedup) still performs comparably with top baselines.
  arXiv  Detail & Related papers  (2025-03-05T17:59:19Z)
• Progressive Growing of Video Tokenizers for Temporally Compact Latent Spaces [20.860632218272094]
  Video tokenizers are essential for latent video diffusion models, converting raw video data into latent spaces for efficient training.<n>We propose an alternative approach to enhance temporal compression.<n>We develop a bootstrapped high-temporal-compression model that progressively trains high-compression blocks atop well-trained lower-compression models.
  arXiv  Detail & Related papers  (2025-01-09T18:55:15Z)
• Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution [65.91317390645163]
  Upscale-A-Video is a text-guided latent diffusion framework for video upscaling. It ensures temporal coherence through two key mechanisms: locally, it integrates temporal layers into U-Net and VAE-Decoder, maintaining consistency within short sequences. It also offers greater flexibility by allowing text prompts to guide texture creation and adjustable noise levels to balance restoration and generation.
  arXiv  Detail & Related papers  (2023-12-11T18:54:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.