HiCrew: Hierarchical Reasoning for Long-Form Video Understanding via Question-Aware Multi-Agent Collaboration

• URL: http://arxiv.org/abs/2604.21444v1
• Date: Thu, 23 Apr 2026 09:04:32 GMT
• Title: HiCrew: Hierarchical Reasoning for Long-Form Video Understanding via Question-Aware Multi-Agent Collaboration
• Abstract summary: We introduce HiCrew, a hierarchical multi-agent framework that addresses limitations through three core contributions.<n>First, we propose a Hybrid Tree structure that preserves temporal topology while performing relevance-guided hierarchical clustering.<n>Second, we develop a Question-Aware Captioning mechanism that synthesizes intent-driven visual prompts to generate semantic descriptions.
• Score: 9.907651803712803
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long-form video understanding remains fundamentally challenged by pervasive spatiotemporal redundancy and intricate narrative dependencies that span extended temporal horizons. While recent structured representations compress visual information effectively, they frequently sacrifice temporal coherence, which is critical for causal reasoning. Meanwhile, existing multi-agent frameworks operate through rigid, pre-defined workflows that fail to adapt their reasoning strategies to question-specific demands. In this paper, we introduce HiCrew, a hierarchical multi-agent framework that addresses these limitations through three core contributions. First, we propose a Hybrid Tree structure that leverages shot boundary detection to preserve temporal topology while performing relevance-guided hierarchical clustering within semantically coherent segments. Second, we develop a Question-Aware Captioning mechanism that synthesizes intent-driven visual prompts to generate precision-oriented semantic descriptions. Third, we integrate a Planning Layer that dynamically orchestrates agent collaboration by adaptively selecting roles and execution paths based on question complexity. Extensive experiments on EgoSchema and NExT-QA validate the effectiveness of our approach, demonstrating strong performance across diverse question types with particularly pronounced gains in temporal and causal reasoning tasks that benefit from our hierarchical structure-preserving design.

Related papers

• Structure over Depth: A Single-Block Spatio-Temporal Transformer for Multi-Entity Reasoning [0.0]
  Multi-entity temporal data requires capturing dependencies across entities, time, and their interactions.<n> Transformer-based approaches decompose well but often rely on deep stacks of layers to learn these heterogeneous dependencies implicitly.<n>We propose a structured-temporal-self-attention transformer block that explicitly models all three within a single stage.<n>We evaluate the approach on video-based group activity recognition, skeleton-based human interaction analysis, and wearable sensor-based activity recognition.
  arXiv  Detail & Related papers  (2026-07-25T07:02:07Z)
• MRMS: A Multi-Resolution Memory Substrate for Long-Lived AI Agents [2.808851478009314]
  Long-lived AI agents require continuity across interactions.<n> Continuity cannot be obtained by simply extending the prompt window.<n>We propose an architectural memory substrate organized along two axes.
  arXiv  Detail & Related papers  (2026-07-06T02:54:37Z)
• Improving Coherence and Persistence in Agentic AI for System Optimization [9.443037059325086]
  Engram organizes exploration into a sequence of agents that iteratively design, test, and analyze mechanisms.<n>We find that Engram exhibits superior performance across diverse domains including multi-cloud multicast inference, LLM request routing, and optimizing KV cache reuse in databases with natural language queries.
  arXiv  Detail & Related papers  (2026-03-22T17:04:50Z)
• AgentLongBench: A Controllable Long Benchmark For Long-Contexts Agents via Environment Rollouts [78.33143446024485]
  We introduce textbfAgentLongBench, which evaluates agents through simulated environment rollouts based on Lateral Thinking Puzzles.<n>This framework generates rigorous interaction trajectories across knowledge-intensive and knowledge-free scenarios.
  arXiv  Detail & Related papers  (2026-01-28T16:05:44Z)
• AMA: Adaptive Memory via Multi-Agent Collaboration [54.490349689939166]
  We propose Adaptive Memory via Multi-Agent Collaboration (AMA), a novel framework that leverages coordinated agents to manage memory across multiple granularities.<n>AMA significantly outperforms state-of-the-art baselines while reducing token consumption by approximately 80% compared to full-context methods.
  arXiv  Detail & Related papers  (2026-01-28T08:09:49Z)
• Adaptive Multi-Agent Reasoning for Text-to-Video Retrieval [12.701443847087164]
  We propose an adaptive multi-agent retrieval framework that orchestrates specialized agents over multiple reasoning iterations.<n>Our framework achieves a twofold improvement over CLIP4Clip and significantly outperforms state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2025-12-02T09:52:51Z)
• COACH: Collaborative Agents for Contextual Highlighting - A Multi-Agent Framework for Sports Video Analysis [29.994841160588397]
  We propose a reconfigurable Multi-Agent System (MAS) as a foundational framework for sports video understanding.<n>In our system, each agent functions as a distinct "cognitive tool" specializing in a specific aspect of analysis.<n>This work presents a paradigm shift towards a flexible, scalable, and interpretable system for robust, cross-task sports video intelligence.
  arXiv  Detail & Related papers  (2025-12-01T16:38:07Z)
• StreamingCoT: A Dataset for Temporal Dynamics and Multimodal Chain-of-Thought Reasoning in Streaming VideoQA [60.86024022291499]
  We introduce StreamingCoT, the first dataset explicitly designed for temporally evolving reasoning in streaming video streams.<n>Our framework generates per-second dense descriptions and constructs temporally-dependent semantic segments through similarity fusion.<n>This dataset establishes a foundation for advancing research in streaming video understanding, complex temporal reasoning, and multimodal inference.
  arXiv  Detail & Related papers  (2025-10-29T09:47:38Z)
• Hollywood Town: Long-Video Generation via Cross-Modal Multi-Agent Orchestration [73.65102758687289]
  This study introduces three innovations to improve multi-agent collaboration.<n>First, we propose OmniAgent, a hierarchical, graph-based multi-agent framework for long video generation.<n>Second, inspired by context engineering, we propose hypergraph nodes that enable temporary group discussions.
  arXiv  Detail & Related papers  (2025-10-25T20:34:18Z)
• Respecting Temporal-Causal Consistency: Entity-Event Knowledge Graphs for Retrieval-Augmented Generation [69.45495166424642]
  We develop a robust and discriminative QA benchmark to measure temporal, causal, and character consistency understanding in narrative documents.<n>We then introduce Entity-Event RAG (E2RAG), a dual-graph framework that keeps separate entity and event subgraphs linked by a bipartite mapping.<n>Across ChronoQA, our approach outperforms state-of-the-art unstructured and KG-based RAG baselines, with notable gains on causal and character consistency queries.
  arXiv  Detail & Related papers  (2025-06-06T10:07:21Z)
• Layer-of-Thoughts Prompting (LoT): Leveraging LLM-Based Retrieval with Constraint Hierarchies [0.3946282433423277]
  Layer-of-Thoughts Prompting (LoT) uses constraint hierarchies to filter and refine candidate responses to a given query. LoT significantly improves the accuracy and comprehensibility of information retrieval tasks.
  arXiv  Detail & Related papers  (2024-10-16T01:20:44Z)
• Multi-granularity Contrastive Cross-modal Collaborative Generation for End-to-End Long-term Video Question Answering [53.39158264785098]
  Long-term Video Question Answering (VideoQA) is a challenging vision-and-language bridging task. We present an entirely end-to-end solution for VideoQA: Multi-granularity Contrastive cross-modal collaborative Generation model.
  arXiv  Detail & Related papers  (2024-10-12T06:21:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.