ProactiveBench: Can Streaming Video Models Really Interact Like Humans?

• URL: http://arxiv.org/abs/2609.12658v1
• Date: Fri, 11 Sep 2026 10:05:42 GMT
• Title: ProactiveBench: Can Streaming Video Models Really Interact Like Humans?
• Abstract summary: Streaming video understanding requires models to process continuous multimodal input while maintaining temporal context.<n>We introduce ProactiveBench, which evaluates models at one-second stream intervals without an explicit response cue.
• Score: 14.13356441344051
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Streaming video understanding requires models to process continuous multimodal input while maintaining temporal context. Existing evaluations are predominantly reactive: they query a model at a selected timestamp and therefore do not assess when it should respond. Proactive interaction instead requires monitoring a standing request, responding within an appropriate interval after the target event, and otherwise remaining silent. We introduce ProactiveBench, which evaluates models at one-second stream intervals without an explicit response cue. Its six subtasks vary trigger ambiguity and timing tolerance. Event Sensitivity geometrically combines response and silence rates on the same recording; four window-based subtasks distinguish early, in-window, and missed responses; and Duplicate Counting penalizes omissions and repetitions. Premature responses outnumber missed responses for four of the six evaluated systems, revealing a substantial gap in the temporal decision-making required for human-like interaction.

Related papers

• Multi-Faceted Interactivity Alignment in Full-Duplex Speech Models [53.470209949659115]
  We propose a post-training alignment method that improves full spoken dialogue models through RL.<n>We address the four canonical axes of interactivity: handling turn-taking, back-channel pause, and user interruption.<n>An extra-based reward for response quality prevents semantic degradation.
  arXiv  Detail & Related papers  (2026-06-09T17:46:55Z)
• OmniPro: A Comprehensive Benchmark for Omni-Proactive Streaming Video Understanding [17.161087166882744]
  We present OmniPro, the first benchmark to jointly evaluate omni-modal perception, proactive responding, and diverse video understanding tasks.<n>It comprises 2,700 human-verified samples spanning 9 sub-tasks and 3 cognitive levels, covering 6 basic video understanding capabilities.<n> Notably, 84% of samples require audio signals (speech or non-speech), and each sample is annotated with modality-isolation labels to enable fine-grained multimodal analysis.
  arXiv  Detail & Related papers  (2026-05-18T15:55:22Z)
• StreamPro: From Reactive Perception to Proactive Decision-Making in Streaming Video [53.82672457255517]
  Proactive streaming video understanding requires models to continuously process video streams and decide when to respond, rather than merely what to respond.<n>Existing benchmarks largely follow a "see-then-answer" paradigm, where responses are triggered only after explicit evidence appears.<n>We introduce StreamPro, a new benchmark that evaluates streaming models from three complementary perspectives: Perception Understanding, Temporal Reasoning, and Proactive Agency.
  arXiv  Detail & Related papers  (2026-05-11T05:01:15Z)
• Chronological Thinking in Full-Duplex Spoken Dialogue Language Models [66.84843878538207]
  Chronological Thinking aims to improve response quality in full SDLMs.<n>No additional latency: once the user stops speaking, the agent halts thinking and begins speaking without further delay.<n>Results: Experiments demonstrate the effectiveness of chronological thinking through both objective metrics and human evaluations.
  arXiv  Detail & Related papers  (2025-10-02T10:28:11Z)
• From What to Respond to When to Respond: Timely Response Generation for Open-domain Dialogue Agents [26.437011114518917]
  TimelyChat benchmark evaluates the capabilities of language models to predict appropriate time intervals and generate time-conditioned responses.<n>We construct a large-scale training dataset by leveraging unlabeled event knowledge from a temporal commonsense knowledge graph.<n>We then train Timer, a dialogue agent designed to proactively predict time intervals and generate timely responses that align with those intervals.
  arXiv  Detail & Related papers  (2025-06-17T07:56:32Z)
• Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reaction [81.34648970317383]
  We present Dispider, a system that disentangles Perception, Decision, and Reaction.<n>Experiments show that Dispider not only maintains strong performance in conventional video QA tasks, but also significantly surpasses previous online models in streaming scenario responses.
  arXiv  Detail & Related papers  (2025-01-06T18:55:10Z)
• Answering Ambiguous Questions via Iterative Prompting [84.3426020642704]
  In open-domain question answering, due to the ambiguity of questions, multiple plausible answers may exist. One approach is to directly predict all valid answers, but this can struggle with balancing relevance and diversity. We present AmbigPrompt to address the imperfections of existing approaches to answering ambiguous questions.
  arXiv  Detail & Related papers  (2023-07-08T04:32:17Z)
• Generating Dialogue Responses from a Semantic Latent Space [75.18449428414736]
  We propose an alternative to the end-to-end classification on vocabulary. We learn the pair relationship between the prompts and responses as a regression task on a latent space. Human evaluation showed that learning the task on a continuous space can generate responses that are both relevant and informative.
  arXiv  Detail & Related papers  (2020-10-04T19:06:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.