Em-Garde: A Propose-Match Framework for Proactive Streaming Video Understanding

• URL: http://arxiv.org/abs/2603.19054v1
• Date: Thu, 19 Mar 2026 15:47:50 GMT
• Title: Em-Garde: A Propose-Match Framework for Proactive Streaming Video Understanding
• Abstract summary: Em-Garde is a novel framework that decouples semantic understanding from streaming perception.<n>Experiments on StreamingBench and OVO-Bench demonstrate consistent improvements over prior models in proactive response accuracy and efficiency.
• Score: 19.76957611582713
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in Streaming Video Understanding has enabled a new interaction paradigm where models respond proactively to user queries. Current proactive VideoLLMs rely on per-frame triggering decision making, which suffers from an efficiency-accuracy dilemma. We propose Em-Garde, a novel framework that decouples semantic understanding from streaming perception. At query time, the Instruction-Guided Proposal Parser transforms user queries into structured, perceptually grounded visual proposals; during streaming, a Lightweight Proposal Matching Module performs efficient embedding-based matching to trigger responses. Experiments on StreamingBench and OVO-Bench demonstrate consistent improvements over prior models in proactive response accuracy and efficiency, validating an effective solution for proactive video understanding under strict computational constraints.

Related papers

• StreamPro: From Reactive Perception to Proactive Decision-Making in Streaming Video [53.82672457255517]
  Proactive streaming video understanding requires models to continuously process video streams and decide when to respond, rather than merely what to respond.<n>Existing benchmarks largely follow a "see-then-answer" paradigm, where responses are triggered only after explicit evidence appears.<n>We introduce StreamPro, a new benchmark that evaluates streaming models from three complementary perspectives: Perception Understanding, Temporal Reasoning, and Proactive Agency.
  arXiv  Detail & Related papers  (2026-05-11T05:01:15Z)
• Query-Conditioned Evidential Keyframe Sampling for MLLM-Based Long-Form Video Understanding [21.306038832082553]
  Multimodal Large Language Models (MLLMs) have shown strong performance on video question answering.<n>Their application to long-form videos is constrained by limited context length and computational cost.<n>We propose an evidence-driven sampling framework grounded in information theory.
  arXiv  Detail & Related papers  (2026-04-01T15:02:50Z)
• STRIDE: When to Speak Meets Sequence Denoising for Streaming Video Understanding [77.20037111885226]
  Real-world deployments increasingly require streaming perception and proactive interaction.<n>In this work, we revisit proactive activation in streaming video as a structured sequence modeling problem.<n>We propose STRIDE, which employs a lightweight masked diffusion module at the activation interface to jointly predict and progressively refine activation signals.
  arXiv  Detail & Related papers  (2026-03-29T09:23:45Z)
• Video-QTR: Query-Driven Temporal Reasoning Framework for Lightweight Video Understanding [37.682165829414494]
  Video-QTR is a lightweight framework that redefines video comprehension as a query-guided reasoning process.<n>We show that Video-QTR achieves state-of-the-art performance while reducing input frame consumption by up to 73%.
  arXiv  Detail & Related papers  (2025-12-10T06:28:00Z)
• StreamAgent: Towards Anticipatory Agents for Streaming Video Understanding [52.55809460075286]
  We propose a StreamAgent that anticipates the temporal intervals and spatial regions expected to contain future task-relevant information.<n>We integrate question semantics and historical observations through prompting the anticipatory agent to anticipate the temporal progression of key events.<n>Our method outperforms existing methods in response accuracy and real-time efficiency, highlighting its practical value for real-world streaming scenarios.
  arXiv  Detail & Related papers  (2025-08-03T18:15:42Z)
• ReAgent-V: A Reward-Driven Multi-Agent Framework for Video Understanding [71.654781631463]
  ReAgent-V is a novel agentic video understanding framework.<n>It integrates efficient frame selection with real-time reward generation during inference.<n>Extensive experiments on 12 datasets demonstrate significant gains in generalization and reasoning.
  arXiv  Detail & Related papers  (2025-06-02T04:23:21Z)
• Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reaction [81.34648970317383]
  We present Dispider, a system that disentangles Perception, Decision, and Reaction.<n>Experiments show that Dispider not only maintains strong performance in conventional video QA tasks, but also significantly surpasses previous online models in streaming scenario responses.
  arXiv  Detail & Related papers  (2025-01-06T18:55:10Z)
• AntPivot: Livestream Highlight Detection via Hierarchical Attention Mechanism [64.70568612993416]
  We formulate a new task Livestream Highlight Detection, discuss and analyze the difficulties listed above and propose a novel architecture AntPivot to solve this problem. We construct a fully-annotated dataset AntHighlight to instantiate this task and evaluate the performance of our model.
  arXiv  Detail & Related papers  (2022-06-10T05:58:11Z)
• FineDiving: A Fine-grained Dataset for Procedure-aware Action Quality Assessment [93.09267863425492]
  We argue that understanding both high-level semantics and internal temporal structures of actions in competitive sports videos is the key to making predictions accurate and interpretable. We construct a new fine-grained dataset, called FineDiving, developed on diverse diving events with detailed annotations on action procedures.
  arXiv  Detail & Related papers  (2022-04-07T17:59:32Z)
• Video Super-Resolution with Recurrent Structure-Detail Network [120.1149614834813]
  Most video super-resolution methods super-resolve a single reference frame with the help of neighboring frames in a temporal sliding window. We propose a novel recurrent video super-resolution method which is both effective and efficient in exploiting previous frames to super-resolve the current frame.
  arXiv  Detail & Related papers  (2020-08-02T11:01:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.