Persistent Object Narratives for Token-Efficient Video Language Models

• URL: http://arxiv.org/abs/2608.04866v1
• Date: Wed, 05 Aug 2026 13:59:04 GMT
• Title: Persistent Object Narratives for Token-Efficient Video Language Models
• Abstract summary: Video large language models (Video-LLMs) have made strong progress in open-ended video understanding.<n>We introduce SlotNarrative, a slot-based interface that organizes a video into persistent object narratives represented by compact object-state tokens.
• Score: 10.96152392810444
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Video large language models (Video-LLMs) have made strong progress in open-ended video understanding. However, their visual interfaces remain token-intensive and provide limited explicit structure for linking recurring object evidence across time. We introduce SlotNarrative, a slot-based interface that organizes a video into persistent object narratives represented by compact object-state tokens. Rather than compressing frame-wise features before establishing temporal correspondence, SlotNarrative first groups visual features into object-like slots and then associates recurring observations with clip-level object entries through a lightweight, parameter-free memory that integrates multiple complementary matching cues. Each retained entry is serialized into two token types: an identity token that summarizes persistent object appearance and a set of state tokens that encode segment-level appearance, geometry, visibility, and trajectory information. This design yields an interface of only 144 allocated visual-token positions for a frozen Video-LLM, independent of the number of sampled frames. Across multiple datasets, SlotNarrative achieves a favorable trade-off between accuracy and visual-token count compared with prior compact Video-LLM interfaces. Experimental results establish persistent object narratives as a compact, structured, and temporally organized visual interface for Video-LLMs. Our code will be made publicly available.

Related papers

• Advancing Narrative Long Video Generation via Training-Free Identity-Aware Memory [79.01059178883817]
  IAMFlow is a training-free identity-aware memory framework that explicitly models and tracks persistent entity identities.<n>A VLM asynchronously verifies and refines attributes from rendered frames, enabling explicit entity tracking in place of implicit similarity-based matching.<n>NarraStream-Bench is a benchmark for narrative streaming video generation that features 324 multi-prompt scripts spanning six dimensions.
  arXiv  Detail & Related papers  (2026-05-18T17:54:34Z)
• BoxTuning: Directly Injecting the Object Box for Multimodal Model Fine-Tuning [18.859227273698615]
  BoxTuning injects object spatial-temporal information directly into the visual modality.<n>It reduces the token cost significantly, achieving 87-93% text token reduction in practice.<n>It also preserves full temporal resolution, where the trajectory trails further encode inter-frame motion direction and speed within each.
  arXiv  Detail & Related papers  (2026-04-13T07:49:31Z)
• AlcheMinT: Fine-grained Temporal Control for Multi-Reference Consistent Video Generation [58.844504598618094]
  We propose AlcheMinT, a unified framework that introduces explicit timestamps conditioning for subject-driven video generation.<n>Our approach introduces a novel positional encoding mechanism that unlocks the encoding of temporal intervals, associated in our case with subject identities.<n>We incorporate subject-descriptive text tokens to strengthen binding between visual identity and video captions, mitigating ambiguity during generation.
  arXiv  Detail & Related papers  (2025-12-11T18:59:34Z)
• MomentSeg: Moment-Centric Sampling for Enhanced Video Pixel Understanding [40.37010049965347]
  Referring Video Object (RefVOS) seeks to segment target objects in videos guided by natural language descriptions.<n>We propose a unified framework that jointly optimize Temporal Sentence Grounding (TSG) and RefVOS, naturally incorporating key moment grounding capability.
  arXiv  Detail & Related papers  (2025-10-10T11:18:21Z)
• VideoAnchor: Reinforcing Subspace-Structured Visual Cues for Coherent Visual-Spatial Reasoning [69.64660280965971]
  VideoAnchor is a plug-and-play module that leverages subspace affinities to reinforce visual cues across frames without retraining.<n>We show consistent performance gains on benchmarks with InternVL2-8B and Q2.5VL-72B.<n>Our codes will be made public at https://github.com/feufhd/VideoAnchor.
  arXiv  Detail & Related papers  (2025-09-29T17:54:04Z)
• SimToken: A Simple Baseline for Referring Audio-Visual Segmentation [29.88252418748085]
  Referring Audio-Visual (Ref-AVS) aims to segment specific objects in videos based on natural language expressions.<n>This task poses significant challenges in cross-modal reasoning and fine-grained object localization.<n>We propose a framework, SimToken, that integrates a multimodal large language model (MLLM) with the Segment Anything Model (SAM)
  arXiv  Detail & Related papers  (2025-09-22T08:55:04Z)
• Memory-efficient Streaming VideoLLMs for Real-time Procedural Video Understanding [51.91097761028129]
  We introduce ProVideLLM, an end-to-end framework for real-time procedural video understanding.<n>ProVideLLM integrates a multimodal cache configured to store two types of tokens.<n>By interleaving these tokens in our multimodal cache, ProVideLLM ensures sub-linear scaling of memory and compute with video length.
  arXiv  Detail & Related papers  (2025-04-10T17:13:08Z)
• VrdONE: One-stage Video Visual Relation Detection [30.983521962897477]
  Video Visual Relation Detection (VidVRD) focuses on understanding how entities over time and space in videos. Traditional methods for VidVRD, challenged by its complexity, typically split the task into two parts: one for identifying what relation are present and another for determining their temporal boundaries. We propose VrdONE, a streamlined yet efficacious one-stage model for VidVRD.
  arXiv  Detail & Related papers  (2024-08-18T08:38:20Z)
• Training-Free Robust Interactive Video Object Segmentation [82.05906654403684]
  We propose a training-free prompt tracking framework for interactive video object segmentation (I-PT) We jointly adopt sparse points and boxes tracking, filtering out unstable points and capturing object-wise information. Our framework has demonstrated robust zero-shot video segmentation results on popular VOS datasets.
  arXiv  Detail & Related papers  (2024-06-08T14:25:57Z)
• Exploring Motion and Appearance Information for Temporal Sentence Grounding [52.01687915910648]
  We propose a Motion-Appearance Reasoning Network (MARN) to solve temporal sentence grounding. We develop separate motion and appearance branches to learn motion-guided and appearance-guided object relations. Our proposed MARN significantly outperforms previous state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2022-01-03T02:44:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.