CAM: Question Answering on Entity-Centric Videos with Continuous Extraction and Adaptive Querying

• URL: http://arxiv.org/abs/2609.06504v1
• Date: Sun, 06 Sep 2026 09:39:49 GMT
• Title: CAM: Question Answering on Entity-Centric Videos with Continuous Extraction and Adaptive Querying
• Abstract summary: Memory facilitates question answering over long videos by extracting and retrieving facts.<n>Existing solutions typically extract independent memory entries from fixed-length video clips.<n>We propose CAM, featuring continuous extraction for high-level semantics and adaptive querying for fine-grained details.
• Score: 11.228743250589902
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Memory facilitates question answering over long videos by extracting and retrieving facts to fit within the limited context windows of multimodal LLMs (MLLMs). Existing solutions typically extract independent memory entries from fixed-length video clips and thus cannot capture high-level semantics that need to be summarized over extended time periods, such as character traits and relations. Moreover, they rely solely on similarity-based retrieval and may fail to retrieve the fine-grained details required for question answering. To tackle these problems, we propose CAM, featuring continuous extraction for high-level semantics and adaptive querying for fine-grained details. In particular, CAM stores the entities and relations extracted from video clips in a knowledge graph. To capture the high-level semantics of each entity or relation, CAM summarizes the local subgraph of the target entity or relation once the subgraph reaches a predefined size. To retrieve the fine-grained details required for question answering, CAM supports multiple search methods, including knowledge graph traversal, video re-watching, and audio listening. It utilizes a planner-executor-verifier pipeline to adaptively compose these search methods according to question intent. Evaluations on three benchmarks show that CAM outperforms SOTA baselines and improves their accuracy by up to 23 percentage points. Code is available at https://github.com/Jake-Tian/CAM.

Related papers

• Fewer Steps, Better Performance: Efficient Cross-Modal Clip Trimming for Video Moment Retrieval Using Language [60.91064560080974]
  We propose an efficient approach, SpotVMR, to trim the query-relevant clip.<n>Our proposed SpotVMR can serve as plug-and-play module, which achieves efficiency for state-of-the-art VMR methods.
  arXiv  Detail & Related papers  (2026-05-28T11:42:28Z)
• ForeSea: AI Forensic Search with Multi-modal Queries for Video Surveillance [56.15563109738998]
  ForeSea is an AI forensic search system with a 3-stage, plug-and-play pipeline.<n>ForeSea improves accuracy by 3.5% and temporal IoU by 11.0 over prior VideoRAG models.
  arXiv  Detail & Related papers  (2026-03-24T07:15:28Z)
• VideoDetective: Clue Hunting via both Extrinsic Query and Intrinsic Relevance for Long Video Understanding [40.699624658181456]
  VideoDetective is a framework that integrates query-to-segment relevance and inter-segment affinity for effective clue hunting in long-video question answering.<n>Our method consistently achieves substantial gains across a wide range of mainstream MLLMs on representative benchmarks, with accuracy improvements of up to 7.5% on VideoMME-long.
  arXiv  Detail & Related papers  (2026-03-23T17:59:51Z)
• MAGNET: A Multi-agent Framework for Finding Audio-Visual Needles by Reasoning over Multi-Video Haystacks [67.31276358668424]
  We introduce a novel task named AV-HaystacksQA, where the goal is to identify salient segments across different videos in response to a query and link them together to generate the most informative answer.<n> AVHaystacks is an audio-visual benchmark comprising 3100 annotated QA pairs designed to assess the capabilities of LMMs in multi-video retrieval and temporal grounding task.<n>We propose a model-agnostic, multi-agent framework to address this challenge, achieving up to 89% and 65% relative improvements over baseline methods on BLEU@4 and GPT evaluation scores in QA task on our proposed AVHaystack
  arXiv  Detail & Related papers  (2025-06-08T06:34:29Z)
• CoT-RVS: Zero-Shot Chain-of-Thought Reasoning Segmentation for Videos [59.391265901911005]
  We propose CoT-RVS, a novel framework employing the zero-shot Chain-of-Thought (CoT) capability of MLLM to address complex challenges by temporal-semantic reasoning.<n>CoT-RVS analyzes the visible objects within a given frame that possibly match the language query (semantic), and chooses a corresponding for each object that can be observed effortlessly among all frames (temporal)<n>Our framework's training-free feature further allows its extension to process online video streams, where the CoT is used at test time to update the object of interest when a better target starts to emerge
  arXiv  Detail & Related papers  (2025-05-24T07:01:31Z)
• Hallucination Mitigation Prompts Long-term Video Understanding [36.26790392889717]
  This paper constructs a comprehensive hallucination mitigation pipeline based on existing MLLMs. We use the CLIP Score to guide the frame sampling process with questions, selecting key frames relevant to the question. During the answer generation stage, we utilize chain-of-thought and in-context learning techniques to explicitly control the generation of answers.
  arXiv  Detail & Related papers  (2024-06-17T08:44:03Z)
• VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videos [67.78336281317347]
  Long-form understanding is complicated by the high redundancy of video data and the abundance of query-irrelevant information.<n>We propose VideoTree, a training-free framework which builds a query-adaptive and hierarchical video representation for LLM reasoning over long-form videos.
  arXiv  Detail & Related papers  (2024-05-29T15:49:09Z)
• LongVLM: Efficient Long Video Understanding via Large Language Models [55.813206751150716]
  LongVLM is a simple yet powerful VideoLLM for long video understanding. We encode video representations that incorporate both local and global information. Our model produces more precise responses for long video understanding.
  arXiv  Detail & Related papers  (2024-04-04T11:33:29Z)
• TraveLER: A Modular Multi-LMM Agent Framework for Video Question-Answering [48.55956886819481]
  We introduce a modular multi-LMM agent framework based on several agents with different roles. Specifically, we propose TraveLER, a method that can create a plan to "Traverse" through the video. We find that the proposed TraveLER approach improves performance on several VideoQA benchmarks without the need to fine-tune on specific datasets.
  arXiv  Detail & Related papers  (2024-04-01T20:58:24Z)
• GMMFormer: Gaussian-Mixture-Model Based Transformer for Efficient Partially Relevant Video Retrieval [59.47258928867802]
  Given a text query, partially relevant video retrieval (PRVR) seeks to find videos containing pertinent moments in a database. This paper proposes GMMFormer, a Gaussian-Mixture-Model based Transformer which models clip representations implicitly. Experiments on three large-scale video datasets demonstrate the superiority and efficiency of GMMFormer.
  arXiv  Detail & Related papers  (2023-10-08T15:04:50Z)
• Query-Dependent Video Representation for Moment Retrieval and Highlight Detection [8.74967598360817]
  Key objective of MR/HD is to localize the moment and estimate clip-wise accordance level, i.e., saliency score, to a given text query. Recent transformer-based models do not fully exploit the information of a given query. We introduce Query-Dependent DETR (QD-DETR), a detection transformer tailored for MR/HD.
  arXiv  Detail & Related papers  (2023-03-24T09:32:50Z)
• Frame-wise Cross-modal Matching for Video Moment Retrieval [32.68921139236391]
  Video moment retrieval targets at retrieving a moment in a video for a given language query. The challenges of this task include 1) the requirement of localizing the relevant moment in an untrimmed video, and 2) bridging the semantic gap between textual query and video contents. We propose an Attentive Cross-modal Relevance Matching model which predicts the temporal boundaries based on an interaction modeling.
  arXiv  Detail & Related papers  (2020-09-22T10:25:41Z)
• Dense-Caption Matching and Frame-Selection Gating for Temporal Localization in VideoQA [96.10612095576333]
  We propose a video question answering model which effectively integrates multi-modal input sources and finds the temporally relevant information to answer questions. Our model is also comprised of dual-level attention (word/object and frame level), multi-head self-cross-integration for different sources (video and dense captions), and which pass more relevant information to gates. We evaluate our model on the challenging TVQA dataset, where each of our model components provides significant gains, and our overall model outperforms the state-of-the-art by a large margin.
  arXiv  Detail & Related papers  (2020-05-13T16:35:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.