EchoCache: Energy-Guided Cross-Modal Caching for Efficient Audio-Driven Video Generation

• URL: http://arxiv.org/abs/2608.02474v2
• Date: Wed, 12 Aug 2026 03:12:13 GMT
• Title: EchoCache: Energy-Guided Cross-Modal Caching for Efficient Audio-Driven Video Generation
• Abstract summary: We propose EchoCache, an energy-guided cross-modal caching framework for efficient audio-driven video generation.<n>Experiments show that EchoCache consistently improves the latency-quality trade-off while preserving generation quality and audio-visual consistency.
• Score: 20.708887969060513
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Audio-driven video generation (A2V) has achieved promising progress in synthesizing temporally coherent and audio-visually aligned videos, yet its inference remains expensive due to the iterative denoising process of diffusion models. Existing caching methods mainly exploit temporal redundancy in visual features while overlooking the cross-modal alignment of A2V, where audio drives visual generation with highly non-uniform temporal importance. In this paper, we identify two levels of misalignment in existing A2V caching methods: temporal-semantic and computation-storage misalignment. To address them, we propose EchoCache, an energy-guided cross-modal caching framework for efficient A2V generation. EchoCache leverages audio time-frequency energy as a saliency anchor to guide latent-level cache updates and further introduces a dynamic timestep-latent caching mechanism with quantized cache management for joint efficiency and memory optimization. Extensive experiments on mainstream A2V models show that EchoCache consistently improves the latency-quality trade-off while preserving generation quality and audio-visual consistency. In particular, on Wan2.2-S2V over the EMTD benchmark, EchoCache achieves a 2.46x speedup with the best overall performance. Code is available at https://github.com/IF-LAB-PKU/EchoCache.

Related papers

• OmniCache: Multidimensional Hierarchical Feature Caching For Diffusion Models [8.344012950413108]
  High-resolution image and video diffusion models, including SD3, FLUX, and recent video diffusion transformers, have substantially improved generative quality but remain expensive at inference time.<n>We address this inefficiency by exploiting redundancy in intermediate diffusion features rather than changing model weights or retraining.<n>We propose OmniCache, a unified hierarchical caching framework that performs multidimensional feature reuse.
  arXiv  Detail & Related papers  (2026-07-26T21:14:48Z)
• SyncCache: Exploiting Asymmetric Dynamics for Fast Audio-Driven Portrait Animation [72.09280113664535]
  Diffusion Transformers (DiTs) have significantly advanced audio-driven portrait animation, but their high computational cost leads to substantial inference latency.<n>We propose SyncCache, a training-free caching acceleration method tailored for DiT-based portrait animation.
  arXiv  Detail & Related papers  (2026-06-29T19:26:13Z)
• Motion-Aware Caching for Efficient Autoregressive Video Generation [73.27762884730272]
  MotionCache exploits inter-frame differences as a lightweight proxy for pixel-level motion characteristics.<n>Experiments on state-of-the-art models like SkyReels-V2 and MAGI-1 demonstrate that MotionCache achieves significant speedups.
  arXiv  Detail & Related papers  (2026-05-03T05:49:27Z)
• WorldCache: Content-Aware Caching for Accelerated Video World Models [50.7543797435026]
  We present textbfWorldCache, a Perception-Constrained Dynamic CaCache framework.<n>WorldCache improves both when and how to reuse features.<n>On Cosmos-2.5-2B evaluated on PAI-Bench, WorldCache achieves textbf$2.3times$ inference speedup while preserving textbf99.4% of baseline quality.
  arXiv  Detail & Related papers  (2026-03-23T17:59:54Z)
• PreciseCache: Precise Feature Caching for Efficient and High-fidelity Video Generation [35.47114707080758]
  High computational costs and slow inference hinder the practical application of video generation models.<n>We propose textbfPreciseCache, a plug-and-play framework that precisely detects and skips truly redundant computations.
  arXiv  Detail & Related papers  (2026-03-01T08:08:49Z)
• SenCache: Accelerating Diffusion Model Inference via Sensitivity-Aware Caching [75.02865981328509]
  Caching reduces computation by reusing previously computed model outputs across timesteps.<n>We propose Sensitivity-Aware Caching (SenCache), a dynamic caching policy that adaptively selects caching timesteps on a per-sample basis.<n>SenCache achieves better visual quality than existing caching methods under similar computational budgets.
  arXiv  Detail & Related papers  (2026-02-27T17:36:09Z)
• Flow caching for autoregressive video generation [72.10021661412364]
  We present FlowCache, the first caching framework specifically designed for autoregressive video generation.<n>Our method achieves remarkable speedups of 2.38 times on MAGI-1 and 6.7 times on SkyReels-V2, with negligible quality degradation.
  arXiv  Detail & Related papers  (2026-02-11T13:11:04Z)
• ERTACache: Error Rectification and Timesteps Adjustment for Efficient Diffusion [30.897215456167753]
  Diffusion models suffer from substantial computational overhead due to their inherently iterative inference process.<n>We propose ERTACache, a principled caching framework that jointly rectifies both error types.<n>ERTACache achieves up to 2x inference speedup while consistently preserving or even improving visual quality.
  arXiv  Detail & Related papers  (2025-08-27T10:37:24Z)
• MixCache: Mixture-of-Cache for Video Diffusion Transformer Acceleration [15.22288174114487]
  Caching is a widely adopted optimization method in DiT models.<n>We propose MixCache, a training-free caching-based framework for efficient video DiT inference.
  arXiv  Detail & Related papers  (2025-08-18T07:49:33Z)
• dKV-Cache: The Cache for Diffusion Language Models [53.85291644298835]
  Diffusion Language Models (DLMs) have been seen as a promising competitor for autoregressive language models.<n>We propose a KV-cache-like mechanism, delayed KV-Cache, for the denoising process of DLMs.<n>Our approach is motivated by the observation that different tokens have distinct representation dynamics throughout the diffusion process.
  arXiv  Detail & Related papers  (2025-05-21T17:32:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.