Gaussian Mixture Modeling for Event-Aware Visual Allocation in Long Video Understanding

• URL: http://arxiv.org/abs/2607.12557v1
• Date: Tue, 14 Jul 2026 09:27:25 GMT
• Title: Gaussian Mixture Modeling for Event-Aware Visual Allocation in Long Video Understanding
• Abstract summary: Large Vision-Language Models (LVLMs) face significant challenges in long video understanding due to the excessive computational cost and information loss associated with uniform sampling.<n>We propose GMM-EVA, which leverages Gaussian Mixture Models to model event-level structure from discrete frame-wise.<n>A differentiated allocation strategy is then applied to preserve one primary high-resolution per event for high-fidelity detail, while utilizing lower-resolution secondary observations to maintain temporal context and optimize token budgets.
• Score: 55.04091863981529
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Vision-Language Models (LVLMs) face significant challenges in long video understanding due to the excessive computational cost and information loss associated with uniform sampling. Existing keyframe selection methods often treat video frames as atomic entities and allocate visual budgets equally, thereby overlooking high-level semantic structures and introducing substantial redundancy. To address these limitations, we propose GMM-EVA (Gaussian Mixture Modeling for Event-Aware Visual Allocation), which leverages Gaussian Mixture Models to model event-level structure from discrete frame-wise observations. A differentiated allocation strategy is then applied to preserve one primary high-resolution keyframe per event for high-fidelity detail, while utilizing lower-resolution secondary keyframes to maintain temporal context and optimize token budgets. GMM-EVA is a training-free, plug-and-play framework that generalizes robustly across various relevance measures and downstream LVLMs. Extensive experiments on multiple long video benchmarks demonstrate that our method significantly outperforms uniform sampling. Notably, GMM-EVA achieves comparable performance to baseline selection methods while utilizing only approximately half of the visual token budget, highlighting its superior efficiency and effectiveness.

Related papers

• Query-Conditioned Evidential Keyframe Sampling for MLLM-Based Long-Form Video Understanding [21.306038832082553]
  Multimodal Large Language Models (MLLMs) have shown strong performance on video question answering.<n>Their application to long-form videos is constrained by limited context length and computational cost.<n>We propose an evidence-driven sampling framework grounded in information theory.
  arXiv  Detail & Related papers  (2026-04-01T15:02:50Z)
• GIFT: Global Irreplaceability Frame Targeting for Efficient Video Understanding [63.62445065977448]
  GIFT: Global Irreplaceability Frame Targeting is a training-free framework that selects frames by assessing their intrinsic irreplaceability.<n>We show that GIFT achieves a maximum average improvement of 12.5% across long-form video benchmarks on LLaVA-Video-7B compared to uniform sampling.
  arXiv  Detail & Related papers  (2026-03-26T06:21:41Z)
• MURE: Hierarchical Multi-Resolution Encoding via Vision-Language Models for Visual Document Retrieval [111.99106496142474]
  Visual Document Retrieval (VDR) requires representations that capture both fine-grained visual details and global document structure.<n>Existing VDR models struggle to balance effectiveness and efficiency when processing high-resolution documents.<n>We propose a new X-VisEmb paradigm that progresses from multi-resolution sampling and encoding, through cross-granularity feature fusion, to adaptive representation distillation.
  arXiv  Detail & Related papers  (2026-03-07T15:17:22Z)
• Twin Restricted Kernel Machines for Multiview Classification [10.208788616684162]
  Multi-view learning (MVL) focuses on improving generalization performance by leveraging complementary information from multiple perspectives or views.<n>We introduce the multiview twin restricted kernel machine (TMvRKM), a novel model that integrates the strengths of kernel machines with the multiview framework.<n>The proposed TMvRKM model is rigorously tested on UCI, KEEL, and AwA benchmark datasets.
  arXiv  Detail & Related papers  (2025-12-12T03:54:19Z)
• DUAL-VAD: Dual Benchmarks and Anomaly-Focused Sampling for Video Anomaly Detection [8.294763803639391]
  Video Anomaly Detection (VAD) is critical for surveillance and public safety.<n>Existing benchmarks are limited to either frame-level or video-level tasks.<n>This work introduces a softmax-based frame allocation strategy that prioritizes anomaly-dense segments while maintaining full-video coverage.
  arXiv  Detail & Related papers  (2025-09-15T05:48:22Z)
• CronusVLA: Towards Efficient and Robust Manipulation via Multi-Frame Vision-Language-Action Modeling [84.51372201195132]
  CronusVLA is a unified framework that extends single-frame VLA models to the multi-frame paradigm.<n>CronusVLA achieves leading performance and superior robustness, with a 70.9% success rate.<n>These results highlight the potential of efficient multi-frame adaptation in VLA models for more powerful and robust real-world deployment.
  arXiv  Detail & Related papers  (2025-06-24T17:30:27Z)
• ViaRL: Adaptive Temporal Grounding via Visual Iterated Amplification Reinforcement Learning [68.76048244253582]
  We introduce ViaRL, the first framework to leverage rule-based reinforcement learning (RL) for optimizing frame selection in video understanding.<n>ViaRL utilizes the answer accuracy of a downstream model as a reward signal to train a frame selector through trial-and-error.<n>ViaRL consistently delivers superior temporal grounding performance and robust generalization across diverse video understanding tasks.
  arXiv  Detail & Related papers  (2025-05-21T12:29:40Z)
• Selective State Space Memory for Large Vision-Language Models [0.0]
  State Space Memory Integration (SSMI) is a novel approach for efficient fine-tuning of LVLMs.<n>SSMI captures long-range dependencies and injects task-specific visual and sequential patterns effectively.<n> experiments on benchmark datasets, including COCO Captioning, VQA, and Flickr30k, demonstrate that SSMI achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-12-13T05:40:50Z)
• Cauchy-Schwarz Regularized Autoencoder [68.80569889599434]
  Variational autoencoders (VAE) are a powerful and widely-used class of generative models. We introduce a new constrained objective based on the Cauchy-Schwarz divergence, which can be computed analytically for GMMs. Our objective improves upon variational auto-encoding models in density estimation, unsupervised clustering, semi-supervised learning, and face analysis.
  arXiv  Detail & Related papers  (2021-01-06T17:36:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.