PCA: Persistence-Aware Compression and Aggregation for Fast Video Large Language Models

• URL: http://arxiv.org/abs/2607.22726v1
• Date: Wed, 22 Jul 2026 05:17:38 GMT
• Title: PCA: Persistence-Aware Compression and Aggregation for Fast Video Large Language Models
• Abstract summary: This paper introduces a training-free $mathbfP$ersistence-Aware $mathbfC$ompression and $mathbfA$ggregation (PCA) method to preserve high-sampling raw visual information before the encoding stage.<n>PCA consistently outperforms existing state-of-the-art approaches in both efficiency and accuracy.
• Score: 63.31550146180189
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite advances in Video Large Language Models (VLLMs) that have displayed promising outcomes in video understanding, the redundancy in the long-duration frames remains a hindrance to efficient reasoning. This paper introduces a training-free $\mathbf{P}$ersistence-Aware $\mathbf{C}$ompression and $\mathbf{A}$ggregation (PCA) method designed to preserve high-fidelity raw visual information before the encoding stage. PCA can be built on arbitrary VLLMs and consists of two modules: 1) A Dynamic Downsampling (DD) module that adaptively removes redundant frames by analyzing frame-wise similarity. 2) A Persistence-Aware Motion Enhancement (PAME) module that enriches each selected keyframe by aggregating the temporal context of its neighbors, ensuring that essential information is preserved even after aggressive frame reduction. Our approach substantially reduces the computation of long-context modeling, while enhancing the performance of the baseline model. Extensive experiments demonstrate that PCA consistently outperforms existing state-of-the-art approaches in both efficiency and accuracy, achieving a speedup of 1.8$\times$ to 2.5$\times$ compared to the baseline VLLM. The code is open-sourced at https://github.com/Heisenberg10110/PCA.

Related papers

• Fast-Decoding Diffusion Language Models via Progress-Aware Confidence Schedules [25.251683954675958]
  We present SchED, a training-free, model-agnostic early-exit algorithm.<n>SchED aggregates full-span logit margins and halts decoding once a smooth, progress-dependent confidence threshold is met.<n>We show that SchED is robust and clearly outperforms prior confidence-based early-exit methods.
  arXiv  Detail & Related papers  (2025-12-02T16:01:08Z)
• PPMStereo: Pick-and-Play Memory Construction for Consistent Dynamic Stereo Matching [51.98089287914147]
  textbfPick-and-textbflay textbfMemory (PM) construction module for dynamic bfStereo matching, dubbed as bftextPPMStereo.<n>Inspired by the two-stage decision-making process in humans, we propose a textbfPick-and-textbflay textbfMemory (PM) construction module for dynamic bfStereo matching, dubbed as bftextPPMStereo.
  arXiv  Detail & Related papers  (2025-10-23T03:52:39Z)
• SpecVLM: Fast Speculative Decoding in Vision-Language Models [14.243294546325714]
  Speculative decoding is a powerful way to accelerate autoregressive large language models (LLMs)<n>We study speculative decoding for vision-language models (VLMs)<n>We introduce SpecVLM, a practical system that delivers 1.5--2.3x end-to-end speedups over full autoregressive inference.
  arXiv  Detail & Related papers  (2025-09-15T11:53:56Z)
• LiftVSR: Lifting Image Diffusion to Video Super-Resolution via Hybrid Temporal Modeling with Only 4$\times$RTX 4090s [16.456543112614586]
  Diffusion models have advanced video super-resolution (VSR) by enhancing perceptual quality.<n>We propose LiftVSR, an efficient VSR framework that leverages and mitigates the image-wise diffusion prior to PixArt-$alpha$, achieving state-of-the-art results.<n>Experiments on several typical VSR benchmarks have demonstrated that LiftVSR achieves impressive performance with significantly lower computational costs.
  arXiv  Detail & Related papers  (2025-06-10T07:49:33Z)
• Towards Scalable Modeling of Compressed Videos for Efficient Action Recognition [6.168286187549952]
  We propose a hybrid end-to-end framework that factorizes learning across three key concepts to reduce inference cost by $330times$ versus prior art.<n> Experiments show that our method results in a lightweight architecture achieving state-of-the-art video recognition performance.
  arXiv  Detail & Related papers  (2025-03-17T21:13:48Z)
• STORM: Token-Efficient Long Video Understanding for Multimodal LLMs [116.4479155699528]
  STORM is a novel architecture incorporating a dedicated temporal encoder between the image encoder and the Video-LLMs.<n>We show that STORM achieves state-of-the-art results across various long video understanding benchmarks.
  arXiv  Detail & Related papers  (2025-03-06T06:17:38Z)
• TFMQ-DM: Temporal Feature Maintenance Quantization for Diffusion Models [52.454274602380124]
  Diffusion models heavily depend on the time-step $t$ to achieve satisfactory multi-round denoising. We propose a Temporal Feature Maintenance Quantization (TFMQ) framework building upon a Temporal Information Block. Powered by the pioneering block design, we devise temporal information aware reconstruction (TIAR) and finite set calibration (FSC) to align the full-precision temporal features.
  arXiv  Detail & Related papers  (2023-11-27T12:59:52Z)
• Can SAM Boost Video Super-Resolution? [78.29033914169025]
  We propose a simple yet effective module -- SAM-guidEd refinEment Module (SEEM) This light-weight plug-in module is specifically designed to leverage the attention mechanism for the generation of semantic-aware feature. We apply our SEEM to two representative methods, EDVR and BasicVSR, resulting in consistently improved performance with minimal implementation effort.
  arXiv  Detail & Related papers  (2023-05-11T02:02:53Z)
• Approximated Bilinear Modules for Temporal Modeling [116.6506871576514]
  Two-layers in CNNs can be converted to temporal bilinear modules by adding an auxiliary-branch sampling. Our models can outperform most state-of-the-art methods on SomethingSomething v1 and v2 datasets without pretraining.
  arXiv  Detail & Related papers  (2020-07-25T09:07:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.