GRACE: Boosting Video MLLMs with Grounded Action-Centric Evidence for Viewer Sentiment Prediction

• URL: http://arxiv.org/abs/2606.16198v1
• Date: Mon, 15 Jun 2026 04:12:43 GMT
• Title: GRACE: Boosting Video MLLMs with Grounded Action-Centric Evidence for Viewer Sentiment Prediction
• Abstract summary: We propose a grounded action-centric evidence augmentation framework that enhances video MLLMs' clue extraction and comprehension.<n>Our method extracts temporally ordered subject-verb-object triplets and auxiliary visible textual cues from action-centric video descriptions.<n> Experiments on the Pitts dataset show consistent improvements over Qwen2.5-VL and Qwen3-VL baselines.
• Score: 24.816669216202246
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Viewer sentiment prediction in video advertisements aims to infer the latent affective response evoked in the audience. To bridge the gap between what is shown and what is felt, models must deduce hidden viewer emotions from explicit visual narratives, concrete character-object interactions, and visible textual cues. However, standard Multimodal Large Language Models (MLLMs) typically rely on holistic frame representations, which leave these fine-grained, affect-relevant events implicit and complicate precise emotional reasoning. To address this, we propose a grounded action-centric evidence augmentation framework that enhances video MLLMs' clue extraction and comprehension by introducing explicit event structure and localized visual evidence. Our method extracts temporally ordered subject-verb-object (SVO) triplets and auxiliary visible textual cues from action-centric video descriptions, grounds subject and object entities as visual entity crops, and then enables the MLLM to perform clue-enhanced emotional reasoning based on these extracted structured clues. In this way, action triplets specify "what happens", while grounded visual entity crops anchor "who or what participates in each event" to concrete visual evidence. Experiments on the Pitts dataset show consistent improvements over Qwen2.5-VL and Qwen3-VL baselines. Ablation studies, cross-dataset evaluation on AdsQA, and transfer experiments on an emotion-focused TVQA subset further support the effectiveness and generalization of our approach.

Related papers

• Leveraging Text-to-Image Diffusion Models for Unsupervised Visual Object Tracking [53.08526613328243]
  Unsupervised visual object tracking is a challenging task that requires following arbitrary targets in videos without training on ground-truth annotations.<n>We build on text-to-image diffusion models to exploit the rich semantic knowledge encoded in pretrained models.<n>Our method Diff-Tracking is composed of two main components: an initial prompt learner and an online prompt updater.
  arXiv  Detail & Related papers  (2026-05-26T12:26:56Z)
• AffectSeek: Agentic Affective Understanding in Long Videos under Vague User Queries [39.694740404232526]
  We study textbfVague-Query-driven video Affective Understanding (VQAU), a new task that requires models to localize affective moments in long videos.<n>We also propose textbfAffectSeek, an agentic framework that actively seeks, verifies, and explains affective moments in long videos.
  arXiv  Detail & Related papers  (2026-05-07T03:47:26Z)
• VideoAnchor: Reinforcing Subspace-Structured Visual Cues for Coherent Visual-Spatial Reasoning [69.64660280965971]
  VideoAnchor is a plug-and-play module that leverages subspace affinities to reinforce visual cues across frames without retraining.<n>We show consistent performance gains on benchmarks with InternVL2-8B and Q2.5VL-72B.<n>Our codes will be made public at https://github.com/feufhd/VideoAnchor.
  arXiv  Detail & Related papers  (2025-09-29T17:54:04Z)
• ImplicitQA: Going beyond frames towards Implicit Video Reasoning [39.63171940350552]
  ImplicitQA is a novel benchmark designed to test VideoQA models on human-like implicit reasoning.<n>ImplicitQA comprises 1K meticulously annotated QA pairs drawn from 1K high-quality creative video clips.
  arXiv  Detail & Related papers  (2025-06-26T19:53:54Z)
• BiMa: Towards Biases Mitigation for Text-Video Retrieval via Scene Element Guidance [10.268638578607977]
  BiMa is a novel framework designed to mitigate biases in both visual and textual representations.<n>For visual debiasing, we integrate these scene elements into the video embeddings, enhancing them to emphasize fine-grained and salient details.<n>For textual debiasing, we introduce a mechanism to disentangle text features into content and bias components, enabling the model to focus on meaningful content.
  arXiv  Detail & Related papers  (2025-06-04T05:40:54Z)
• SPECTRUM: Semantic Processing and Emotion-informed video-Captioning Through Retrieval and Understanding Modalities [0.7510165488300369]
  This paper proposes a novel Semantic Processing and Emotion-informed video-Captioning Through Retrieval and Understanding Modalities (SPECTRUM) framework. SPECTRUM discerns multimodal semantics and emotional themes using Visual Text Attribute Investigation (VTAI) and determines the orientation of descriptive captions. They exploit video-to-text retrieval capabilities and the multifaceted nature of video content to estimate the emotional probabilities of candidate captions.
  arXiv  Detail & Related papers  (2024-11-04T10:51:47Z)
• End-to-end Open-vocabulary Video Visual Relationship Detection using Multi-modal Prompting [68.37943632270505]
  Open-vocabulary video visual relationship detection aims to expand video visual relationship detection beyond categories.<n>Existing methods usually use trajectory detectors trained on closed datasets to detect object trajectories.<n>We propose an open-vocabulary relationship that leverages the rich semantic knowledge of CLIP to discover novel relationships.
  arXiv  Detail & Related papers  (2024-09-19T06:25:01Z)
• HENASY: Learning to Assemble Scene-Entities for Egocentric Video-Language Model [9.762722976833581]
  Current models rely extensively on instance-level alignment between video and language modalities. We take an inspiration from human perception and explore a compositional approach for ego video representation.
  arXiv  Detail & Related papers  (2024-06-01T05:41:12Z)
• VLLMs Provide Better Context for Emotion Understanding Through Common Sense Reasoning [66.23296689828152]
  We leverage the capabilities of Vision-and-Large-Language Models to enhance in-context emotion classification.<n>First, we prompt VLLMs to generate natural language descriptions of the subject's apparent emotion in relation to the visual context.<n>Second, the descriptions, along with the visual input, are used to train a transformer-based architecture.
  arXiv  Detail & Related papers  (2024-04-10T15:09:15Z)
• Reasoning over the Behaviour of Objects in Video-Clips for Adverb-Type Recognition [54.938128496934695]
  We propose a new framework that reasons over object-behaviours extracted from raw-video-clips to recognize the clip's corresponding adverb-types. Specifically, we propose a novel pipeline that extracts human-interpretable object-behaviour-facts from raw video clips. We release two new datasets of object-behaviour-facts extracted from raw video clips.
  arXiv  Detail & Related papers  (2023-07-09T09:04:26Z)
• Spatio-Temporal Graph for Video Captioning with Knowledge Distillation [50.034189314258356]
  We propose a graph model for video captioning that exploits object interactions in space and time. Our model builds interpretable links and is able to provide explicit visual grounding. To avoid correlations caused by the variable number of objects, we propose an object-aware knowledge distillation mechanism.
  arXiv  Detail & Related papers  (2020-03-31T03:58:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.