O-MARC: Omni Memory-Augmented Compression Distillation for Efficient Video Understanding

• URL: http://arxiv.org/abs/2605.26584v1
• Date: Tue, 26 May 2026 06:07:11 GMT
• Title: O-MARC: Omni Memory-Augmented Compression Distillation for Efficient Video Understanding
• Abstract summary: We introduce OMAC, a training free plug in compression method that preserves salient visual memory and temporally grounded audio anchors.<n>We also introduce O-MARC, a compression distillation framework for learning with memory compressed multimodal contexts.<n>On Qwen2.5- Omni-3B, O-MARC improves the average score across four benchmarks to 45.8, outperforming full token inference at 44.1 and OmniZip at 41.0.
• Score: 16.003413095973784
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Omnimodal large language models enable unified audio video understanding, but long joint token sequences make inference costly, and existing benchmarks do not fully isolate audio visual association in noisy user generated videos. We introduce UGC-AVQA, a public UGC benchmark with 1,000 videos and 4,816 QA pairs, where an audio removal test ensures that benchmark questions require both acoustic and visual evidence. To reduce inference cost, we propose OMAC, a training free plug in compression method that preserves salient visual memory and temporally grounded audio anchors. To further make compact models robust to compressed inputs, we introduce O-MARC, a compression distillation framework for learning with memory compressed multimodal contexts. On Qwen2.5-Omni-3B, O-MARC improves the average score across four benchmarks to 45.8, outperforming full token inference at 44.1 and OmniZip at 41.0. OMAC also keeps inference efficient, reducing latency by 34.6\% (1.53$\times$ speedup) and memory by 34.7\% compared with full token inference.

Related papers

• Omni2LoRA: Coherence-Preserving Parametric Memory for Efficient Omni Language Models [65.49950267695267]
  We introduce Omni2LoRA, a framework for efficient parametric memory compression via coherence-preserving context distillation.<n>By converting multimodal memory into a fixed-budget, reusable parameter state, our method drives answer-time multimodal-token load to zero.
  arXiv  Detail & Related papers  (2026-08-10T07:49:10Z)
• Deferred Audio Pruning with Local Audio-Visual Dynamics for Omni-LLMs [13.179456385239481]
  A-Pack is a framework that defers audio pruning until query-conditioned multimodal interactions emerge.<n>A-Pack achieves the strongest average performance among the evaluated prior methods.<n>It reduces prefill FLOPs by up to 78% and improving decoding throughput by up to 2.21x.
  arXiv  Detail & Related papers  (2026-08-09T16:18:41Z)
• VoxZip: Semantic-Anchored Temporal KV Cache Compression for Long-Context Audio Inference [53.883871460875234]
  VoxZip is a semantic-anchored KV cache compression framework.<n>First stage uses automatic speech recognition (ASR) transcriptions as explicit semantic anchors to temporally align, compress, and fuse audio tokens.<n>Second stage employs a dynamic filtering strategy based on temporally decayed accumulated attention to evict non-essential tokens.
  arXiv  Detail & Related papers  (2026-08-09T08:17:18Z)
• OmniFocus: Query-Guided Modality-Balanced Token Compression for Omni-Modal Large Language Models [72.34361439042634]
  We propose a training-free query-guided token compression method for OmniLLMs that performs independent importance estimation for video and audio.<n>Experiments on the Qwen2.5- Omni model family across four audio-visual benchmarks show that OmniFocus maintains strong compressed performance at low token retention ratios.
  arXiv  Detail & Related papers  (2026-07-03T07:41:00Z)
• AVOC: Enhancing Hour-Level Audio-Video Understanding in Omni-Modal LLMs via Retrieval-Inspired Token Compression [22.818321106501788]
  AVOC is a framework for long-form audio-video understanding in Omni-modal Large Language Models.<n>AVOC achieves state-of-the-art performance on long-form audio-video benchmarks.
  arXiv  Detail & Related papers  (2026-06-23T08:06:58Z)
• OmniMem: Perturbation-aware Memory Compression for Streaming Audio-Visual LLMs [30.52254659395564]
  We present OmniMem, a memory-efficient streaming framework for audio-visual large language models (LLMs)<n>Unlike existing compression methods that treat all tokens uniformly, OmniMem introduces a modality-aware memory allocation strategy.<n>We show that OmniMem consistently improves over strong training-free compression baselines by 2-4% absolute accuracy under the same memory budgets.
  arXiv  Detail & Related papers  (2026-05-26T19:12:13Z)
• MuKV: Multi-Grained KV Cache Compression for Long Streaming Video Question-Answering [75.0394545769057]
  KV-caching stores the Key-Value of the historical tokens via LLM prefill.<n>MuKV is a method that features a multi-grained KV cache compression module and a semi-hierarchical retrieval approach.<n> Experiments on long-streaming VideoQA benchmarks show that MuKV significantly improves answer accuracy, without sacrificing memory and online QA efficiency.
  arXiv  Detail & Related papers  (2026-05-21T10:13:03Z)
• OmniRefine: Alignment-Aware Cooperative Compression for Efficient Omnimodal Large Language Models [14.355322423471018]
  We propose a training-free two-stage framework for efficient audio-visual token compression in Omni-LLMs.<n>First, Correspondence-Preserving Chunk Refinement refines native chunk boundaries into cross-modally aligned compression units.<n>Second, Modality-Aware Cooperative Compression jointly compresses video and audio tokens within each refined unit to reduce redundancy while preserving critical evidence.
  arXiv  Detail & Related papers  (2026-05-12T12:42:44Z)
• Open-TQ-Metal: Fused Compressed-Domain Attention for Long-Context LLM Inference on Apple Silicon [0.0]
  We present Open-TQ-Metal, the first implementation of fused compressed-domain attention on Apple Silicon.<n>It enables 128K-context inference for Llama 3.1 70B on a single 64GB consumer Mac.<n>Open-TQ-Metal quantizes the KV cache to int4 on the fly and computes attention directly on the compressed representation.
  arXiv  Detail & Related papers  (2026-04-18T10:39:28Z)
• Pushing the Limits of On-Device Streaming ASR: A Compact, High-Accuracy English Model for Low-Latency Inference [5.685908474026849]
  High-quality automatic speech recognition (ASR) on edge devices requires models that jointly optimize accuracy, latency, and memory footprint while operating entirely on CPU without GPU acceleration.<n>We conduct a systematic empirical study of state-of-the-art ASR architectures, encompassing encoder-decoder, transducer, and LLM-based paradigms, evaluated across batch, chunked, and streaming inference modes.<n>Our recommended configuration, the int4 k-quant variant, achieves 8.20% average streaming WER across eight standard benchmarks, running comfortably faster than real-time on CPU with 0.56 s algorithmic latency.
  arXiv  Detail & Related papers  (2026-04-16T00:04:32Z)
• One Token per Highly Selective Frame: Towards Extreme Compression for Long Video Understanding [51.08792182064565]
  Long video understanding is inherently challenging for vision-language models (VLMs) because of the extensive number of frames.<n>With each video frame typically expanding into tens or hundreds of tokens, the limited context length of large language models (LLMs) forces the VLMs to perceive the frames sparsely and lose temporal information.<n>We propose an extreme compression model for long video understanding, named XComp, achieving a significantly larger compression ratio and enabling denser frame sampling.
  arXiv  Detail & Related papers  (2026-04-15T17:59:52Z)
• KV Cache Quantization for Self-Forcing Video Generation: A 33-Method Empirical Study [0.0]
  We present a comprehensive study of KV-cache compression for self-forcing video generation on a Wan2.1-based Self-Forcing stack.<n>Our study covers 33 quantization and cache-policy variants, 610 prompt-level observations, and 63 benchmark-level summaries.<n>We jointly evaluate peak VRAM, runtime, realized compression ratio, VBench imaging quality, BF16-referenced fidelity (SSIM, LPIPS, PSNR), and terminal drift.
  arXiv  Detail & Related papers  (2026-03-29T01:35:16Z)
• PackCache: A Training-Free Acceleration Method for Unified Autoregressive Video Generation via Compact KV-Cache [61.57938553036056]
  We introduce PackCache, a training-free KV-cache management method that compacts the KV cache through three coordinated mechanisms.<n>In terms of efficiency, PackCache accelerates end-to-end generation by 1.7-2.2x on 48-frame long sequences.
  arXiv  Detail & Related papers  (2026-01-07T19:51:06Z)
• OmniZip: Audio-Guided Dynamic Token Compression for Fast Omnimodal Large Language Models [21.542205813984243]
  We present a training-free, audio-visual token-compression framework that optimize multimodal token representation and inference.<n>Specifically, OmniZip first identifies salient audio tokens, then computes an audio retention score for each time group to capture information.<n>For each time window, OmniZip compresses the video tokens using an interleaved-temporal scheme.
  arXiv  Detail & Related papers  (2025-11-18T15:22:32Z)
• AudioMarathon: A Comprehensive Benchmark for Long-Context Audio Understanding and Efficiency in Audio LLMs [53.248502396225724]
  AudioMarathon is a benchmark designed to evaluate both understanding and inference efficiency on long-form audio.<n>We evaluate state-of-the-art LALMs and observe clear performance drops as audio length grows.<n>The results show large gaps across current LALMs and highlight the need for better temporal reasoning.
  arXiv  Detail & Related papers  (2025-10-08T17:50:16Z)
• MiniCache: KV Cache Compression in Depth Dimension for Large Language Models [48.03117580340151]
  Key-Value ( KV) cache stores key-value states of previously generated tokens. The size of the KV cache grows linearly with sequence length, posing challenges for applications requiring long context input and extensive sequence generation. We present a simple yet effective approach, called MiniCache, to compress the KV cache across layers from a novel depth perspective.
  arXiv  Detail & Related papers  (2024-05-23T09:43:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.