VideoAtlas: Navigating Long-Form Video in Logarithmic Compute

• URL: http://arxiv.org/abs/2603.17948v1
• Date: Wed, 18 Mar 2026 17:20:19 GMT
• Title: VideoAtlas: Navigating Long-Form Video in Logarithmic Compute
• Abstract summary: textbfVideoAtlas is a task-agnostic environment to represent video as a hierarchical grid.<n>The hierarchical structure ensures access depth grows only logarithmically with video length.<n>Video-RLM remains the most duration-robust method with minimal accuracy degradation when scaling from 1-hour to 10-hour benchmarks.
• Score: 3.705718227493618
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Extending language models to video introduces two challenges: representation, where existing methods rely on lossy approximations, and long-context, where caption- or agent-based pipelines collapse video into text and lose visual fidelity. To overcome this, we introduce \textbf{VideoAtlas}, a task-agnostic environment to represent video as a hierarchical grid that is simultaneously lossless, navigable, scalable, caption- and preprocessing-free. An overview of the video is available at a glance, and any region can be recursively zoomed into, with the same visual representation used uniformly for the video, intermediate investigations, and the agent's memory, eliminating lossy text conversion end-to-end. This hierarchical structure ensures access depth grows only logarithmically with video length. For long-context, Recursive Language Models (RLMs) recently offered a powerful solution for long text, but extending them to visual domain requires a structured environment to recurse into, which \textbf{VideoAtlas} provides. \textbf{VideoAtlas} as a Markov Decision Process unlocks Video-RLM: a parallel Master-Worker architecture where a Master coordinates global exploration while Workers concurrently drill into assigned regions to accumulate lossless visual evidence. We demonstrate three key findings: (1)~logarithmic compute growth with video duration, further amplified by a 30-60\% multimodal cache hit rate arising from the grid's structural reuse. (2)~environment budgeting, where bounding the maximum exploration depth provides a principled compute-accuracy hyperparameter. (3)~emergent adaptive compute allocation that scales with question granularity. When scaling from 1-hour to 10-hour benchmarks, Video-RLM remains the most duration-robust method with minimal accuracy degradation, demonstrating that structured environment navigation is a viable and scalable paradigm for video understanding.

Related papers

• HPP: Hierarchical Programmatic Probing for Long Video Understanding by Decoupling Perception and Reasoning [0.764671395172401]
  Hierarchical Program Probing (HPP) is a framework that reformulates long video understanding as iterative, programmatic exploration of a hierarchically segmented video.<n>To make probing tractable over long videos, we introduce three components: information--aware hierarchical segmentation, late-interaction semantic retrieval, structured and probing functions for coarse-to-fine temporal localization.
  arXiv  Detail & Related papers  (2026-06-19T20:43:49Z)
• Kwai Keye-VL-2.0 Technical Report [53.82434681649277]
  Keye-VL-2.0 is a multimodal foundation model designed to advance long-video understanding and agentic intelligence.<n>It is the first to adapt DeepSeek Sparse Attention (DSA) to GQA-based multimodal architectures.<n>Cross-Modal Multi-Teacher On-Policy Distillation (MOPD) paired with Context-RL and Video-RL overcomes the algorithmic dilemma of catastrophic forgetting.
  arXiv  Detail & Related papers  (2026-06-09T09:58:08Z)
• TV-RAG: A Temporal-aware and Semantic Entropy-Weighted Framework for Long Video Retrieval and Understanding [14.570869250170139]
  TV-RAG is a training-free architecture that couples temporal alignment with entropy-guided semantics to improve long-video reasoning.<n>By weaving these temporal and semantic signals together, TV-RAG realises a dual-level reasoning routine that can be grafted onto any LVLM without re-training or fine-tuning.
  arXiv  Detail & Related papers  (2025-12-29T14:10:22Z)
• Vgent: Graph-based Retrieval-Reasoning-Augmented Generation For Long Video Understanding [56.45689495743107]
  Vgent is a graph-based retrieval-reasoning-augmented generation framework to enhance LVLMs for long video understanding.<n>We evaluate our framework with various open-source LVLMs on three long-video understanding benchmarks.
  arXiv  Detail & Related papers  (2025-10-15T19:14:58Z)
• AdaVideoRAG: Omni-Contextual Adaptive Retrieval-Augmented Efficient Long Video Understanding [73.60257070465377]
  AdaVideoRAG is a novel framework that adapts retrieval based on query complexity using a lightweight intent classifier.<n>Our framework employs an Omni-Knowledge Indexing module to build hierarchical databases from text (captions, ASR, OCR), visual features, and semantic graphs.<n> Experiments demonstrate improved efficiency and accuracy for long-video understanding, with seamless integration into existing MLLMs.
  arXiv  Detail & Related papers  (2025-06-16T15:18:15Z)
• SALOVA: Segment-Augmented Long Video Assistant for Targeted Retrieval and Routing in Long-Form Video Analysis [52.050036778325094]
  We introduce SALOVA: Segment-Augmented Video Assistant, a novel video-LLM framework designed to enhance the comprehension of lengthy video content.<n>We present a high-quality collection of 87.8K long videos, each densely captioned at the segment level to enable models to capture scene continuity and maintain rich context.<n>Our framework mitigates the limitations of current video-LMMs by allowing for precise identification and retrieval of relevant video segments in response to queries.
  arXiv  Detail & Related papers  (2024-11-25T08:04:47Z)
• Encoding and Controlling Global Semantics for Long-form Video Question Answering [40.129800076300434]
  We introduce a state space layer (SSL) into multi-modal Transformer to efficiently integrate global semantics of the video. Our SSL includes a gating unit to enable controllability over the flow of global semantics into visual representations. To rigorously evaluate long-form videoQA capacity, we construct two new benchmarks Ego-QA and MAD-QA featuring videos of considerably long length.
  arXiv  Detail & Related papers  (2024-05-30T06:10:10Z)
• LongVLM: Efficient Long Video Understanding via Large Language Models [55.813206751150716]
  LongVLM is a simple yet powerful VideoLLM for long video understanding. We encode video representations that incorporate both local and global information. Our model produces more precise responses for long video understanding.
  arXiv  Detail & Related papers  (2024-04-04T11:33:29Z)
• A Simple Recipe for Contrastively Pre-training Video-First Encoders Beyond 16 Frames [57.758863967770594]
  We build on the common paradigm of transferring large-scale, image--text models to video via shallow temporal fusion.<n>We expose two limitations to the approach: (1) decreased spatial capabilities, likely due to poor video--language alignment in standard video datasets, and (2) higher memory consumption, bottlenecking the number of frames that can be processed.
  arXiv  Detail & Related papers  (2023-12-12T16:10:19Z)
• A Hierarchical Multi-Modal Encoder for Moment Localization in Video Corpus [31.387948069111893]
  We show how to identify a short segment in a long video that semantically matches a text query. To tackle this problem, we propose the HierArchical Multi-Modal EncodeR (HAMMER) that encodes a video at both the coarse-grained clip level and the fine-trimmed frame level. We conduct extensive experiments to evaluate our model on moment localization in video corpus on ActivityNet Captions and TVR datasets.
  arXiv  Detail & Related papers  (2020-11-18T02:42:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.