AffectSeek: Agentic Affective Understanding in Long Videos under Vague User Queries

• URL: http://arxiv.org/abs/2605.05640v1
• Date: Thu, 07 May 2026 03:47:26 GMT
• Title: AffectSeek: Agentic Affective Understanding in Long Videos under Vague User Queries
• Abstract summary: We study textbfVague-Query-driven video Affective Understanding (VQAU), a new task that requires models to localize affective moments in long videos.<n>We also propose textbfAffectSeek, an agentic framework that actively seeks, verifies, and explains affective moments in long videos.
• Score: 39.694740404232526
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing affective understanding studies have mainly focused on recognizing emotions from images, audio signals, or pre-cliped video clips, where the affective evidence is already given. This passive and clip-centered setting does not fully reflect real-world scenarios, in which users often interact with long videos and express their needs through natural-language queries. In this paper, we study \textbf{Vague-Query-driven video Affective Understanding (VQAU)}, a new task that requires models to localize affective moments in long videos, predict their emotion categories, and generate evidence-grounded rationales under vague user queries. To support this task, we construct \textbf{VQAU-Bench}, a benchmark that integrates long videos, vague affective queries, temporal clip annotations, emotion labels, and rationale explanations into a unified evaluation framework. VQAU-Bench enables systematic assessment of semantic-temporal-affective alignment, affective moment localization, emotion classification, and rationale generation. To address the multi-step reasoning challenges of VQAU, we further propose \textbf{AffectSeek}, an agentic framework that actively seeks, verifies, and explains affective moments in long videos. AffectSeek decomposes VQAU into intent interpretation, candidate localization, clip verification, emotion reasoning, and rationale generation, and progressively aligns vague user intent with long-video evidence through role-specialized reasoning and cross-stage verification. Experiments show that VQAU remains challenging for existing affective recognition models and single-step vision-language models, while AffectSeek provides a simple yet effective framework for agentic long-video affective understanding.

Related papers

• From Vision to Language: Investigating Causal Information Flow in Multimodal Decision-Making [45.85783584369188]
  We investigate how visual information contributes to language-based decisions.<n>Our results show that visual information is mainly integrated while the model processes the candidate answer options.<n>We identify a distinct pattern in temporal reasoning, suggesting that VLMs struggle to reconstruct sequential information across video frames.
  arXiv  Detail & Related papers  (2026-09-04T13:52:27Z)
• Incentivizing Vision Language Models to Search for Long Video Question Answering [41.69519333811218]
  VSeek transforms long-video question answering (LVQA) into a multi-turn retrieval process.<n>We propose a novel neuro-symbolic approach that bridges open-ended natural language with discrete visual verification.<n>VSeek improves Pass@1 scores by up to 8% and Pass@4 scores by 15% on long-video understanding benchmarks.
  arXiv  Detail & Related papers  (2026-07-03T05:05:59Z)
• GRACE: Boosting Video MLLMs with Grounded Action-Centric Evidence for Viewer Sentiment Prediction [24.816669216202246]
  We propose a grounded action-centric evidence augmentation framework that enhances video MLLMs' clue extraction and comprehension.<n>Our method extracts temporally ordered subject-verb-object triplets and auxiliary visible textual cues from action-centric video descriptions.<n> Experiments on the Pitts dataset show consistent improvements over Qwen2.5-VL and Qwen3-VL baselines.
  arXiv  Detail & Related papers  (2026-06-15T04:12:43Z)
• CACR:Reinforcing Temporal Answer Grounding in Instructional Video via Candidate-Aware Causal Reasoning [8.230031783815749]
  TAGV aims to locate precise video segments that respond to natural language queries.<n>This task remains challenging due to the need to comprehend semantically complex questions.<n>We propose a Candidate-Aware Causal Reasoning framework to tackle these limitations.
  arXiv  Detail & Related papers  (2026-06-07T03:23:18Z)
• FACE-net: Factual Calibration and Emotion Augmentation for Retrieval-enhanced Emotional Video Captioning [81.33341786837974]
  Emotional Video Captioning (EVC) is an emerging task, which aims to describe factual content with the intrinsic emotions expressed in videos.<n>We propose a retrieval-enhanced framework with FActual and Emotion augmentation (FACE-net)<n>FACE-net collaboratively mines factual-emotional semantics and provides adaptive and accurate guidance for generation.
  arXiv  Detail & Related papers  (2026-03-18T07:53:15Z)
• Know-Show: Benchmarking Video-Language Models on Spatio-Temporal Grounded Reasoning [18.15310805625469]
  We present Know-Show, a new benchmark designed to evaluate multimodal Video-Language Models (Video-LMs)<n>Know-Show unifies reasoning and localization within a single evaluation framework consisting of five scenarios across spatial (person, object, person-object, and hand-object) and temporal dimensions.<n>Built from Charades, Action Genome, and Ego4D with 2.5K human-language questions, the benchmark exposes significant gaps between current Video-LMs and human reasoning.<n>To bridge this gap, we propose GRAM, a training-free plug-in that augments Video-LMs with fine-grained grounding
  arXiv  Detail & Related papers  (2025-12-05T08:15:49Z)
• SeViCES: Unifying Semantic-Visual Evidence Consensus for Long Video Understanding [36.30263540665245]
  We propose a framework for effective and reliable long video understanding.<n>SeViCES is training-free and model-agnostic, and introduces two key components.<n> experiments on long video understanding benchmarks show that SeViCES consistently outperforms state-of-the-art methods in both accuracy and robustness.
  arXiv  Detail & Related papers  (2025-10-23T14:55:28Z)
• ImplicitQA: Going beyond frames towards Implicit Video Reasoning [39.63171940350552]
  ImplicitQA is a novel benchmark designed to test VideoQA models on human-like implicit reasoning.<n>ImplicitQA comprises 1K meticulously annotated QA pairs drawn from 1K high-quality creative video clips.
  arXiv  Detail & Related papers  (2025-06-26T19:53:54Z)
• Query-centric Audio-Visual Cognition Network for Moment Retrieval, Segmentation and Step-Captioning [56.873534081386]
  A new topic HIREST is presented, including video retrieval, moment retrieval, moment segmentation, and step-captioning.<n>We propose a query-centric audio-visual cognition network to construct a reliable multi-modal representation for three tasks.<n>This can cognize user-preferred content and thus attain a query-centric audio-visual representation for three tasks.
  arXiv  Detail & Related papers  (2024-12-18T06:43:06Z)
• VideoDistill: Language-aware Vision Distillation for Video Question Answering [24.675876324457747]
  We propose VideoDistill, a framework with language-aware (i.e., goal-driven) behavior in both vision perception and answer generation process. VideoDistill generates answers only from question-related visual embeddings. We conduct experimental evaluations on various challenging video question-answering benchmarks, and VideoDistill achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-04-01T07:44:24Z)
• How Would The Viewer Feel? Estimating Wellbeing From Video Scenarios [73.24092762346095]
  We introduce two large-scale datasets with over 60,000 videos annotated for emotional response and subjective wellbeing. The Video Cognitive Empathy dataset contains annotations for distributions of fine-grained emotional responses, allowing models to gain a detailed understanding of affective states. The Video to Valence dataset contains annotations of relative pleasantness between videos, which enables predicting a continuous spectrum of wellbeing.
  arXiv  Detail & Related papers  (2022-10-18T17:58:25Z)
• Locate before Answering: Answer Guided Question Localization for Video Question Answering [70.38700123685143]
  LocAns integrates a question locator and an answer predictor into an end-to-end model. It achieves state-of-the-art performance on two modern long-term VideoQA datasets.
  arXiv  Detail & Related papers  (2022-10-05T08:19:16Z)
• Dilated Context Integrated Network with Cross-Modal Consensus for Temporal Emotion Localization in Videos [128.70585652795637]
  TEL presents three unique challenges compared to temporal action localization. The emotions have extremely varied temporal dynamics. The fine-grained temporal annotations are complicated and labor-intensive.
  arXiv  Detail & Related papers  (2022-08-03T10:00:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.