Related papers: Video2Reaction: Mapping Video to Audience Reaction Distribution in the Wild

Video2Reaction: Mapping Video to Audience Reaction Distribution in the Wild

URL: http://arxiv.org/abs/2607.06875v1
Date: Wed, 08 Jul 2026 00:17:20 GMT
Title: Video2Reaction: Mapping Video to Audience Reaction Distribution in the Wild
Abstract summary: $textbfVideo2Reaction$ is a dataset that maps short movie segments to a distribution of $textitinduced emotions$ of viewers in the wild.<n>$textbfVideo2Reaction$ spans more than 10,000 videos and serves as a reliable benchmark.<n>We develop a two-stage multi-agent pipeline using only open-source LLMs, achieving 86% correctness under blind human verification.
Score: 12.752010716495205
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
Abstract: Understanding and forecasting audience reactions to video content are crucial for improving content creation, recommendation systems, and media analysis. To enable audience reaction prediction and other content engagement applications, we introduce $\textbf{Video2Reaction}$, a multimodal dataset that maps short movie segments to a distribution of $\textit{induced emotions}$ of viewers in the wild, as expressed through social media. $\textbf{Video2Reaction}$ spans more than 10,000 videos and serves as a reliable benchmark as well as a training resource for audience reaction prediction. To enable cost-effective continuous annotations as reactions may change over time, we develop a two-stage multi-agent pipeline using only open-source LLMs, achieving 86% correctness under blind human verification despite the inherently noisy and subjective nature of the task. We establish the first benchmark for video-to-reaction-distribution prediction in the wild and show that pretrained foundation video models fail in zero-shot settings, while finetuning transforms them into state-of-the-art predictors capable of modeling both full reaction distributions and dominant responses from video alone. However, the task remains challenging: even the strongest methods achieve only 77% Top-3 F1 in dominant reaction prediction (LLaVA-Next), highlighting a substantial gap in modeling collective audience reaction. \modification{Dataset and code are available at our project page: https://information-fusion-lab-umass.github.io/video2reaction-bench.github.io

Related papers

Video2Reaction: Training Foundation Video Models to Predict Audience Reaction [12.752010716495205]
Video2Reaction is a dataset that maps short movie segments to the induced emotional reactions of viewers in the wild.<n>We benchmark two vision-language models (VLMs) finetuned with LoRA.<n>We show that VLMs learn effectively from Video2Reaction and outperform specialized baselines on dominant reaction prediction.
arXiv Detail & Related papers (2026-09-01T19:44:23Z)
Unified Video-Action Joint Denoising for Dexterous Action and Data Generation [13.23337683310307]
We propose Donk, a unified video-action denoising model for dexterous hands.<n>Donk samples future videos and bimanual MANO trajectories as an action policy.<n>Without the image condition, the same denoising architecture samples video-action rollouts from a text-conditioned distribution.
arXiv Detail & Related papers (2026-06-02T16:39:23Z)
From Preferences to Prejudice: The Role of Alignment Tuning in Shaping Social Bias in Video Diffusion Models [69.4332879415364]
We introduce VideoBiasEval, a framework for evaluating social representation in video generation.<n>VideoBiasEval employs an event-based prompting strategy to disentangle semantic content from actor attributes.<n>We conduct the first end-to-end analysis connecting biases in human preference datasets, their amplification in reward models, and their propagation through alignment-tuned video diffusion models.
arXiv Detail & Related papers (2025-10-20T07:37:43Z)
Video-LevelGauge: Investigating Contextual Positional Bias in Large Video Language Models [51.67019924750931]
Video-LevelGauge is a benchmark designed to assess positional bias in large video language models (LVLMs)<n>We employ standardized probes and customized contextual setups, allowing flexible control over context length, probe position, and contextual types.<n>Our benchmark comprises 438 manually curated videos spanning multiple types, yielding 1,177 high-quality multiple-choice questions and 120 open-ended questions.
arXiv Detail & Related papers (2025-08-27T07:58:16Z)
FingER: Content Aware Fine-grained Evaluation with Reasoning for AI-Generated Videos [18.3012265316413]
We propose a novel entity-level reasoning evaluation framework, $textbfF$ine-grained $textbfE$ntity-level questions.<n>Our model surpasses existing methods by a relative margin of $11.8%$ on GenAI-Bench and $5.5%$ on MonetBench with only 3.3k training videos.
arXiv Detail & Related papers (2025-04-14T16:07:16Z)
Delving Deep into Engagement Prediction of Short Videos [34.38399476375175]
This study delves deep into the intricacies of predicting engagement for newly published videos with limited user interactions. We introduce a substantial dataset comprising 90,000 real-world short videos from Snapchat. Our method demonstrates its ability to predict engagements of short videos purely from video content.
arXiv Detail & Related papers (2024-09-30T23:57:07Z)
ViGT: Proposal-free Video Grounding with Learnable Token in Transformer [28.227291816020646]
Video grounding task aims to locate queried action or event in an untrimmed video based on rich linguistic descriptions. Existing proposal-free methods are trapped in complex interaction between video and query. We propose a novel boundary regression paradigm that performs regression token learning in a transformer.
arXiv Detail & Related papers (2023-08-11T08:30:08Z)
Causal Video Summarizer for Video Exploration [74.27487067877047]
Causal Video Summarizer (CVS) is proposed to capture the interactive information between the video and query. Based on the evaluation of the existing multi-modal video summarization dataset, experimental results show that the proposed approach is effective.
arXiv Detail & Related papers (2023-07-04T22:52:16Z)
Transform-Equivariant Consistency Learning for Temporal Sentence Grounding [66.10949751429781]
We introduce a novel Equivariant Consistency Regulation Learning framework to learn more discriminative representations for each video. Our motivation comes from that the temporal boundary of the query-guided activity should be consistently predicted. In particular, we devise a self-supervised consistency loss module to enhance the completeness and smoothness of the augmented video.
arXiv Detail & Related papers (2023-05-06T19:29:28Z)
Modality-Balanced Embedding for Video Retrieval [21.81705847039759]
We identify a modality bias phenomenon that the video encoder almost entirely relies on text matching. We propose MBVR (short for Modality Balanced Video Retrieval) with two key components. We show empirically that our method is both effective and efficient in solving modality bias problem.
arXiv Detail & Related papers (2022-04-18T06:29:46Z)
Probabilistic Representations for Video Contrastive Learning [64.47354178088784]
This paper presents a self-supervised representation learning method that bridges contrastive learning with probabilistic representation. By sampling embeddings from the whole video distribution, we can circumvent the careful sampling strategy or transformations to generate augmented views of the clips.
arXiv Detail & Related papers (2022-04-08T09:09:30Z)
Towards Debiasing Temporal Sentence Grounding in Video [59.42702544312366]
temporal sentence grounding in video (TSGV) task is to locate a temporal moment from an untrimmed video, to match a language query. Without considering bias in moment annotations, many models tend to capture statistical regularities of the moment annotations. We propose two debiasing strategies, data debiasing and model debiasing, to "force" a TSGV model to capture cross-modal interactions.
arXiv Detail & Related papers (2021-11-08T08:18:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.