Daily Paper Briefing

Abstract Overview

OneStreamer is a 4B streaming video large language model designed to unify real-time perception, reusable memory formation, and proactive task response in a single causal generation process. The core idea is to jointly learn query-independent evidence recording and task answering, using Proactive Hierarchical Caption Memory (PHCM) to write time-grounded local captions and event summaries, and Proactive State Transition Learning (PSTL) to better supervise response timing without overemphasizing repeated waiting states. The authors also introduce OneStreamer-1M, a synthesized streaming video interaction dataset with more than one million records that aligns both content and response timing with the evidence actually available at each moment. Across eight streaming video benchmarks, the model outperforms compared methods, while ablations indicate that generated caption memory improves historical question answering without degrading real-time perception.

Novelty

The main novelty is treating proactive generation as a unified interface for both query-independent memory writing and user-visible response, rather than handling memory retention and response timing as separate disconnected modules. The paper also contributes two specific mechanisms—PHCM for hierarchical textual memory and PSTL for selective state-token supervision—alongside an evidence-aligned streaming data synthesis pipeline.

Results

Across eight evaluated streaming video benchmarks spanning perception, memory, and proactive response, the 4B OneStreamer model achieves the top reported results among compared methods, outperforming its Qwen3-VL 4B base model and larger baselines such as 11B MOSS-VL-Realtime. Ablations demonstrate that PHCM boosts historical QA while maintaining strong real-time perception, and PSTL outperforms dense state supervision while supervising only 27.5% of annotated state tokens. On a 360-second video sample, PHCM reduces context token count by 93.1% and answer-stage time-to-first-token from 4.560 s to 0.124 s compared to retaining full visual history.

Key Points

1. OneStreamer unifies query-independent evidence recording and proactive task response into a single causal video-language generation framework.
2. Proactive Hierarchical Caption Memory (PHCM) records past observations as time-grounded local captions and event summaries, enabling reuse of distant evidence without revisiting historical visual tokens.
3. Proactive State Transition Learning (PSTL) mitigates the imbalance of repeated silence/waiting states by selectively supervising representative state transitions and persistence while preserving all output anchors.

References

• arXiv: https://arxiv.org/abs/2610.01762v1
• Fugu-MT: https://fugumt.com/fugumt/paper_check/2610.01762v1
• Hugging Face Papers: https://huggingface.co/papers/2610.01762
• Project: https://mcg-nju.github.io/OneStreamer