LayerCache: Exploiting Layer-wise Velocity Heterogeneity for Efficient Flow Matching Inference

• URL: http://arxiv.org/abs/2604.16492v1
• Date: Mon, 13 Apr 2026 15:44:24 GMT
• Title: LayerCache: Exploiting Layer-wise Velocity Heterogeneity for Efficient Flow Matching Inference
• Abstract summary: Flow Matching models achieve state-of-the-art image generation quality but incur substantial inference cost due to iterative denoising through large Transformer networks.<n>We propose LayerCache, a layer-aware caching framework that partitions the Transformer into layer groups and makes independent, per-group caching decisions at each denoising step.<n>On Qwen-Image (1024x1024, 50 steps), LayerCache achieves PSNR 37.46 dB (+5.38 dB over MeanCache), SSIM 0.9834, and LPIPS 0.0178 (a 70% reduction over MeanCache) at 1.37x speedup.
• Score: 11.772150619675527
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Flow Matching models achieve state-of-the-art image generation quality but incur substantial inference cost due to iterative denoising through large Transformer networks. We observe that different layer groups within a Transformer exhibit markedly heterogeneous velocity dynamics: shallow layers are highly stable and amenable to aggressive caching, while deep layers undergo large velocity changes that demand full computation. Existing caching methods, however, treat the entire Transformer as a monolithic unit, applying a single caching decision per timestep and thus failing to exploit this heterogeneity. Based on this finding, we propose LayerCache, a layer-aware caching framework that partitions the Transformer into layer groups and makes independent, per-group caching decisions at each denoising step. LayerCache introduces an adaptive JVP span K selection mechanism that leverages per-group stability measurements to balance estimation accuracy and computational savings. We formulate a three-dimensional scheduling problem over timesteps, layer groups, and JVP span, and solve it with a greedy budget allocation algorithm. On Qwen-Image (1024x1024, 50 steps), LayerCache achieves PSNR 37.46 dB (+5.38 dB over MeanCache), SSIM 0.9834, and LPIPS 0.0178 (a 70% reduction over MeanCache) at 1.37x speedup, dominating all prior caching methods on the quality-speed Pareto frontier.

Related papers

• OmniCache: Multidimensional Hierarchical Feature Caching For Diffusion Models [8.344012950413108]
  High-resolution image and video diffusion models, including SD3, FLUX, and recent video diffusion transformers, have substantially improved generative quality but remain expensive at inference time.<n>We address this inefficiency by exploiting redundancy in intermediate diffusion features rather than changing model weights or retraining.<n>We propose OmniCache, a unified hierarchical caching framework that performs multidimensional feature reuse.
  arXiv  Detail & Related papers  (2026-07-26T21:14:48Z)
• LaCache: Exact Caching and Precision-Adaptive Inference for Diffusion Large Language Models [9.146969118551167]
  DLLMs enable parallel generation via Semi-Autoregressive (SAR) decoding in text generation.<n>Current methods suffer from severe operator-level redundancy.<n>We propose LaCache, a training-free acceleration framework.
  arXiv  Detail & Related papers  (2026-07-16T10:49:15Z)
• WorldCache: Content-Aware Caching for Accelerated Video World Models [50.7543797435026]
  We present textbfWorldCache, a Perception-Constrained Dynamic CaCache framework.<n>WorldCache improves both when and how to reuse features.<n>On Cosmos-2.5-2B evaluated on PAI-Bench, WorldCache achieves textbf$2.3times$ inference speedup while preserving textbf99.4% of baseline quality.
  arXiv  Detail & Related papers  (2026-03-23T17:59:54Z)
• Frequency-Aware Error-Bounded Caching for Accelerating Diffusion Transformers [11.772150619675527]
  Diffusion Transformers (DiTs) have emerged as the dominant architecture for high-quality image and video generation.<n>Existing caching methods accelerate DiTs by reusing intermediate computations across timesteps, but they share a common limitation: treating the denoising process as uniform across time,depth, and feature dimensions.<n>We propose SpectralCache, a unified caching framework comprising Timestep-Aware Dynamic Scheduling (TADS), Cumulative Error Budgets (CEB), and Frequency-Decomposed Caching (FDC)
  arXiv  Detail & Related papers  (2026-03-05T15:58:06Z)
• SenCache: Accelerating Diffusion Model Inference via Sensitivity-Aware Caching [75.02865981328509]
  Caching reduces computation by reusing previously computed model outputs across timesteps.<n>We propose Sensitivity-Aware Caching (SenCache), a dynamic caching policy that adaptively selects caching timesteps on a per-sample basis.<n>SenCache achieves better visual quality than existing caching methods under similar computational budgets.
  arXiv  Detail & Related papers  (2026-02-27T17:36:09Z)
• MeanCache: From Instantaneous to Average Velocity for Accelerating Flow Matching Inference [11.934900617930774]
  MeanCache is a training-free caching framework for efficient Flow Matching inference.<n>We demonstrate that MeanCache achieves 4.12X and 4.56X and 3.59X acceleration, respectively, while consistently outperforming state-of-the-art caching baselines in generation quality.
  arXiv  Detail & Related papers  (2026-01-27T08:35:50Z)
• LLMCache: Layer-Wise Caching Strategies for Accelerated Reuse in Transformer Inference [0.0]
  Transformer-based language models have achieved remarkable performance across a wide range of tasks, yet their high inference latency poses a significant challenge for real-timeand large-scale deployment.<n>We present LLMCache, a novel layer-wise caching framework that accelerates transformer inference by reusing intermediate activations based on semantic similarity of input sequences.
  arXiv  Detail & Related papers  (2025-12-18T18:18:57Z)
• DiCache: Let Diffusion Model Determine Its Own Cache [62.954717254728166]
  DiCache is a training-free adaptive caching strategy for accelerating diffusion models at runtime.<n>Online Probe Profiling Scheme leverages a shallow-layer online probe to obtain an on-the-fly indicator for the caching error in real time.<n> Dynamic Cache Trajectory Alignment approximates the deep-layer feature output from multi-step historical caches.
  arXiv  Detail & Related papers  (2025-08-24T13:30:00Z)
• MixCache: Mixture-of-Cache for Video Diffusion Transformer Acceleration [15.22288174114487]
  Caching is a widely adopted optimization method in DiT models.<n>We propose MixCache, a training-free caching-based framework for efficient video DiT inference.
  arXiv  Detail & Related papers  (2025-08-18T07:49:33Z)
• MagCache: Fast Video Generation with Magnitude-Aware Cache [91.2771453279713]
  We introduce a novel and robust discovery: a unified magnitude law observed across different models and prompts.<n>We introduce a Magnitude-aware Cache (MagCache) that adaptively skips unimportant timesteps using an error modeling mechanism and adaptive caching strategy.<n> Experimental results show that MagCache achieves 2.10x-2.68x speedups on Open-Sora, CogVideoX, Wan 2.1, and HunyuanVideo, while preserving superior visual fidelity.
  arXiv  Detail & Related papers  (2025-06-10T17:59:02Z)
• QuantCache: Adaptive Importance-Guided Quantization with Hierarchical Latent and Layer Caching for Video Generation [84.91431271257437]
  Diffusion Transformers (DiTs) have emerged as a dominant architecture in video generation.<n>DiTs come with significant drawbacks, including increased computational and memory costs.<n>We propose QuantCache, a novel training-free inference acceleration framework.
  arXiv  Detail & Related papers  (2025-03-09T10:31:51Z)
• Learning-to-Cache: Accelerating Diffusion Transformer via Layer Caching [56.286064975443026]
  We make an interesting and somehow surprising observation: the computation of a large proportion of layers in the diffusion transformer, through a caching mechanism, can be readily removed even without updating the model parameters. We introduce a novel scheme, named Learningto-Cache (L2C), that learns to conduct caching in a dynamic manner for diffusion transformers. Experimental results show that L2C largely outperforms samplers such as DDIM and DPM-r, alongside prior cache-based methods at the same inference speed.
  arXiv  Detail & Related papers  (2024-06-03T18:49:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.