ObjectStream: Latent Objects as Memory Anchors for Streaming Video Understanding

• URL: http://arxiv.org/abs/2607.28312v2
• Date: Sat, 01 Aug 2026 16:00:11 GMT
• Title: ObjectStream: Latent Objects as Memory Anchors for Streaming Video Understanding
• Abstract summary: ObjectStream is a training-free framework that treats latent objects as memory anchors for streaming video understanding.<n>It preserves three complementary forms of evidence: persistent object histories, transient object changes, and recent visual context.<n>Experiments on online streaming and offline long-video benchmarks demonstrate both effectiveness and efficiency.
• Score: 50.17161313335395
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Streaming video understanding requires models to continuously retain useful visual evidence before future questions are known. Existing approaches primarily manage the growing visual context according to token importance, temporal redundancy, or segment-level relevance, but rarely organize evidence around objects that persist and evolve over time. Thus, in this paper, we introduce ObjectStream, a training-free framework that treats latent objects as memory anchors for streaming video understanding. ObjectStream induces spatially coherent latent objects directly from frozen Video-LLM representations, links them across frames into persistent anchors, and maintains their histories under a bounded memory budget, without requiring external object detectors or segmentation models. Built on these anchors, ObjectStream preserves three complementary forms of evidence: persistent object histories, transient object changes, and recent visual context. This design enables existing Video Large Language Models (Video-LLMs) to reason over object identities, interactions, and state changes while leaving the underlying model unchanged. Extensive experiments on online streaming and offline long-video benchmarks demonstrate both effectiveness and efficiency. In online streaming evaluation, ObjectStream improves Qwen2.5-VL-7B by 10.0 points on OVO-Bench Real-Time Visual Perception, while reducing peak GPU mem-ory and TTFT by approximately 50%. On offline long-video benchmarks, it surpasses the full-token baseline while discarding 82.5% of visual tokens. These results highlight latent objects as a practical and effective organizing principle for compact streaming video memory.

Related papers

• Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understanding [64.34948094316685]
  We introduce LatentStream, a progressive latent working memory framework that shifts streaming memory from store-and-retrieve to retrieve-and-internalize.<n>LatentStream achieves new state-of-the-art results on existing online and offline video benchmarks.
  arXiv  Detail & Related papers  (2026-09-03T17:28:14Z)
• StreamFlow: Dynamic Memory Flows for Streaming Video Understanding [65.10324190760602]
  StreamFlow is an efficient visual memory framework that enables dynamic, on-demand access to historical visual information.<n>It achieves state-of-the-art streaming video understanding performance, reaching 67.73% overall accuracy on StreamingBench.<n>Relative to the vanilla setting, it improves the visual attention score (VAS) by 59.1% while reducing end-to-end latency and peak memory by 50.4% and 21.1%, respectively.
  arXiv  Detail & Related papers  (2026-08-11T14:19:03Z)
• FOLIO: Focused Semantic Memory for Streaming Video Understanding [7.067883491096352]
  In online streaming video understanding, a video stream continues to arrive and queries may be issued at any time.<n>As this history grows with the stream, memory cost increases and many redundant visual details are retained.<n>We introduce FOLIO, a training-free focused semantic memory system that records important parts of the stream in higher detail.
  arXiv  Detail & Related papers  (2026-07-14T22:09:48Z)
• LiveStarPro: Proactive Streaming Video Understanding with Hierarchical Memory for Long-Horizon Streams [59.485485426790966]
  We introduce LiveStarPro, a live streaming assistant that is designed for proactive video understanding over long-horizon streams.<n>LiveStarPro consistently surpasses existing methods, attaining a 28.9% improvement in semantic correctness.<n>Its streaming key-value cache further yields a 1.58x inference speedup over the same model without caching.
  arXiv  Detail & Related papers  (2026-06-16T11:18:05Z)
• SlotMemory: Object-Centric KV Memory for Streaming Long-Video Generation [30.157534925157844]
  SlotMemory is an object-centric Key-Value memory mechanism for streaming video diffusion.<n>Our approach shifts the memory abstraction from "when" an event occurred to "what" is being represented by decomposing the transformer's key-value manifold.
  arXiv  Detail & Related papers  (2026-05-29T09:04:26Z)
• Thinking in Streaming Video [30.61790766076081]
  ThinkStream is a framework for streaming video reasoning based on a Watch--Think--Speak paradigm.<n> Reasoning-Compressed Streaming Memory (RCSM) treats intermediate reasoning traces as compact semantic memory.<n>Experiments on multiple streaming video benchmarks show that ThinkStream significantly outperforms existing online video models.
  arXiv  Detail & Related papers  (2026-03-13T12:33:36Z)
• Fast SAM2 with Text-Driven Token Pruning [52.8350457627401]
  Segment Anything Model 2 (SAM2), a vision computation model has significantly advanced in prompt-driven video object segmentation.<n>SAM2 pipelines propagate all visual tokens produced by the image encoder through downstream temporal reasoning modules, regardless of their relevance to the target object.<n>We introduce a text-guided token pruning framework that improves inference efficiency by selectively reducing token density prior to temporal propagation.
  arXiv  Detail & Related papers  (2025-12-24T18:59:05Z)
• StreamForest: Efficient Online Video Understanding with Persistent Event Memory [37.73273040737155]
  StreamForest is designed for streaming video understanding.<n>Fine-grained Spatiotemporal Window captures detailed short-term visual cues to improve current scene perception.<n>OnlineIT significantly boosts MLLM performance in both real-time perception and future prediction.
  arXiv  Detail & Related papers  (2025-09-29T14:53:57Z)
• Latest Object Memory Management for Temporally Consistent Video Instance Segmentation [17.805350735895114]
  We present Latest Object Memory Management (LOMM) for temporally consistent video instance segmentation.<n>LOMM robustly tracks and updates the latest states of objects by explicitly modeling their presence in each frame.<n>We introduce Decoupled Object Association (DOA), a strategy that separately handles newly appearing and already existing objects.
  arXiv  Detail & Related papers  (2025-07-26T02:52:41Z)
• Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streams [78.72965584414368]
  We present Flash-VStream, a video-language model that simulates the memory mechanism of human. Compared to existing models, Flash-VStream achieves significant reductions in latency inference and VRAM consumption. We propose VStream-QA, a novel question answering benchmark specifically designed for online video streaming understanding.
  arXiv  Detail & Related papers  (2024-06-12T11:07:55Z)
• Region Aware Video Object Segmentation with Deep Motion Modeling [56.95836951559529]
  Region Aware Video Object (RAVOS) is a method that predicts regions of interest for efficient object segmentation and memory storage. For efficient segmentation, object features are extracted according to the ROIs, and an object decoder is designed for object-level segmentation. For efficient memory storage, we propose motion path memory to filter out redundant context by memorizing the features within the motion path of objects between two frames.
  arXiv  Detail & Related papers  (2022-07-21T01:44:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.