Free Energy-Based Modeling of Emotional Dynamics in Video Advertisements

• URL: http://arxiv.org/abs/2601.00812v1
• Date: Mon, 22 Dec 2025 06:51:41 GMT
• Title: Free Energy-Based Modeling of Emotional Dynamics in Video Advertisements
• Abstract summary: We have quantified "pleasantness," "surprise," and "habituation" solely from scene-level expression features of advertising videos.<n>This work can be extended by integrating a wider range of expression elements and validating the approach through subjective ratings.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Emotional responses during advertising video viewing are recognized as essential for understanding media effects because they have influenced attention, memory, and purchase intention. To establish a methodological basis for explainable emotion estimation without relying on external information such as physiological signals or subjective ratings, we have quantified "pleasantness," "surprise," and "habituation" solely from scene-level expression features of advertising videos, drawing on the free energy(FE) principle, which has provided a unified account of perception, learning, and behavior. In this framework, Kullback-Leibler divergence (KLD) has captured prediction error, Bayesian surprise (BS) has captured belief updates, and uncertainty (UN) has reflected prior ambiguity, and together they have formed the core components of FE. Using 1,059 15 s food video advertisements, the experiments have shown that KLD has reflected "pleasantness" associated with brand presentation, BS has captured "surprise" arising from informational complexity, and UN has reflected "surprise" driven by uncertainty in element types and spatial arrangements, as well as by the variability and quantity of presented elements. This study also identified three characteristic emotional patterns, namely uncertain stimulus, sustained high emotion, and momentary peak and decay, demonstrating the usefulness of the proposed method. Robustness across nine hyperparameter settings and generalization tests with six types of Japanese advertising videos (three genres and two durations) confirmed that these tendencies remained stable. This work can be extended by integrating a wider range of expression elements and validating the approach through subjective ratings, ultimately guiding the development of technologies that can support the creation of more engaging advertising videos.

Related papers

• E$^3$mo-Bench: A Scalable Benchmark for Multimodal Evoked and Expressed Emotion Understanding via Bayesian Pairwise Alignment [41.40354084103659]
  We introduce E$3$mo-Bench, a scalable benchmark for understanding evoked and expressed emotions.<n>It evaluates evoked and expressed emotion understanding via $3$ complementary tasks: emotion perception, open-vocabulary recognition, and valence-arousal-dominance (VAD) assessment.<n>We also develop E$3$mo-Score, a training-free agent that aggregates complementary judgments from a five-model committee to improve VAD estimation.
  arXiv  Detail & Related papers  (2026-08-11T11:05:22Z)
• AffectVerse: Emotional World Models for Multimodal Affective Computing [56.144242722718985]
  We propose AffectVerse, an action-free representation-level module for short-horizon latent affective prediction.<n>EWM contains three modules: 1) Cross-Modal Temporal Imagination predicts future video/audio representations from past tokens with multi-step rollout.<n>EWM compresses imagined tokens into modality-aware belief tokens.<n>These results suggest predictive belief-state modeling is a practical alternative for affective computing.
  arXiv  Detail & Related papers  (2026-05-19T15:05:00Z)
• ESI-Bench: Towards Embodied Spatial Intelligence that Closes the Perception-Action Loop [55.468404995694975]
  We develop a benchmark for embodied spatial intelligence spanning 10 task categories and 29 subcategories built on OmniGibson.<n>We conduct experiments on state-of-the-art MLLMs and find that active exploration substantially outperforms passive counterparts.<n>Unlike humans who seek falsifying viewpoints and revise beliefs under contradiction, models commit prematurely with high confidence regardless of evidence quality.
  arXiv  Detail & Related papers  (2026-05-18T17:59:02Z)
• Modeling Induced Pleasure through Cognitive Appraisal Prediction via Multimodal Fusion [0.6397750672942423]
  This study introduces a novel computational model to infer video-induced pleasure via cognitive appraisal variables.<n>The proposed model addresses four challenges: (1) noisy and inconsistent human labels, (2) the semantic gap between "positive emotions" and "pleasure," (3) the scarcity of pleasure-specific datasets, and (4) the limited interpretability of existing black-box fusion methods.<n> Experimental results validate the efficacy of the proposed method in detecting video-induced pleasure, achieving a peak accuracy of 0.6624 in predicting pleasure levels.
  arXiv  Detail & Related papers  (2026-04-26T15:07:41Z)
• FACE-net: Factual Calibration and Emotion Augmentation for Retrieval-enhanced Emotional Video Captioning [81.33341786837974]
  Emotional Video Captioning (EVC) is an emerging task, which aims to describe factual content with the intrinsic emotions expressed in videos.<n>We propose a retrieval-enhanced framework with FActual and Emotion augmentation (FACE-net)<n>FACE-net collaboratively mines factual-emotional semantics and provides adaptive and accurate guidance for generation.
  arXiv  Detail & Related papers  (2026-03-18T07:53:15Z)
• ADEPT: RL-Aligned Agentic Decoding of Emotion via Evidence Probing Tools -- From Consensus Learning to Ambiguity-Driven Emotion Reasoning [67.22219034602514]
  We introduce ADEPT (Agentic Decoding of Emotion via Evidence Probing Tools), a framework that reframes emotion recognition as a multi-turn inquiry process.<n> ADEPT transforms an SLLM into an agent that maintains an evolving candidate emotion set and adaptively invokes dedicated semantic and acoustic probing tools.<n>We show that ADEPT improves primary emotion accuracy in most settings while substantially improving minor emotion characterization.
  arXiv  Detail & Related papers  (2026-02-13T08:33:37Z)
• A Descriptive Model for Modelling Attacker Decision-Making in Cyber-Deception [0.0]
  This paper presents a descriptive model that incorporates the psychological and strategic elements shaping this decision.<n>The framework provides a structured method for analysing engagement decisions in cyber-deception scenarios.
  arXiv  Detail & Related papers  (2025-12-03T10:23:33Z)
• Emotion-Coherent Reasoning for Multimodal LLMs via Emotional Rationale Verifier [53.55996102181836]
  We propose the Emotional Rationale Verifier (ERV) and an Explanation Reward.<n>Our method guides the model to produce reasoning that is explicitly consistent with the target emotion.<n>We show that our approach not only enhances alignment between explanation and prediction but also empowers MLLMs to deliver emotionally coherent, trustworthy interactions.
  arXiv  Detail & Related papers  (2025-10-27T16:40:17Z)
• DeceptionBench: A Comprehensive Benchmark for AI Deception Behaviors in Real-world Scenarios [57.327907850766785]
  characterization of deception across realistic real-world scenarios remains underexplored.<n>We establish DeceptionBench, the first benchmark that systematically evaluates how deceptive tendencies manifest across different domains.<n>On the intrinsic dimension, we explore whether models exhibit self-interested egoistic tendencies or sycophantic behaviors that prioritize user appeasement.<n>We incorporate sustained multi-turn interaction loops to construct a more realistic simulation of real-world feedback dynamics.
  arXiv  Detail & Related papers  (2025-10-17T10:14:26Z)
• RoboView-Bias: Benchmarking Visual Bias in Embodied Agents for Robotic Manipulation [67.38036090822982]
  We propose RoboView-Bias, the first benchmark specifically designed to quantify visual bias in robotic manipulation.<n>We create 2,127 task instances that enable robust measurement of biases induced by individual visual factors and their interactions.<n>Our results highlight that systematic analysis of visual bias is a prerequisite for developing safe and reliable general-purpose embodied agents.
  arXiv  Detail & Related papers  (2025-09-26T13:53:25Z)
• Towards Context-Aware Emotion Recognition Debiasing from a Causal Demystification Perspective via De-confounded Training [14.450673163785094]
  Context-Aware Emotion Recognition (CAER) provides valuable semantic cues for recognizing the emotions of target persons. Current approaches invariably focus on designing sophisticated structures to extract perceptually critical representations from contexts. We present a Contextual Causal Intervention Module (CCIM) to de-confound the confounder.
  arXiv  Detail & Related papers  (2024-07-06T05:29:02Z)
• Towards A Robust Group-level Emotion Recognition via Uncertainty-Aware Learning [27.284076257222942]
  Group-level emotion recognition (GER) is an inseparable part of human behavior analysis.<n>We propose an uncertainty-aware learning (UAL) method to extract more robust representations for GER.<n>We develop an image enhancement module to enhance the model's robustness against severe noise.
  arXiv  Detail & Related papers  (2023-10-06T15:05:41Z)
• Memory in Plain Sight: Surveying the Uncanny Resemblances of Associative Memories and Diffusion Models [65.08133391009838]
  generative process of Diffusion Models (DMs) has recently set state-of-the-art on many AI generation benchmarks. We introduce a novel perspective to describe DMs using the mathematical language of memory retrieval from the field of energy-based Associative Memories (AMs) We present a growing body of evidence that records DMs exhibiting empirical behavior we would expect from AMs, and conclude by discussing research opportunities that are revealed by understanding DMs as a form of energy-based memory.
  arXiv  Detail & Related papers  (2023-09-28T17:57:09Z)
• Affective Image Content Analysis: Two Decades Review and New Perspectives [132.889649256384]
  We will comprehensively review the development of affective image content analysis (AICA) in the recent two decades. We will focus on the state-of-the-art methods with respect to three main challenges -- the affective gap, perception subjectivity, and label noise and absence. We discuss some challenges and promising research directions in the future, such as image content and context understanding, group emotion clustering, and viewer-image interaction.
  arXiv  Detail & Related papers  (2021-06-30T15:20:56Z)
• Affect2MM: Affective Analysis of Multimedia Content Using Emotion Causality [84.69595956853908]
  We present Affect2MM, a learning method for time-series emotion prediction for multimedia content. Our goal is to automatically capture the varying emotions depicted by characters in real-life human-centric situations and behaviors.
  arXiv  Detail & Related papers  (2021-03-11T09:07:25Z)
• Automatic Expansion of Domain-Specific Affective Models for Web Intelligence Applications [3.0012517171007755]
  Sentic computing relies on well-defined affective models of different complexity. The most granular affective model combined with sophisticated machine learning approaches may not fully capture an organisation's strategic positioning goals. This article introduces expansion techniques for affective models, combining common and commonsense knowledge available in knowledge graphs with language models and affective reasoning.
  arXiv  Detail & Related papers  (2021-02-01T13:32:35Z)
• Unboxing Engagement in YouTube Influencer Videos: An Attention-Based Approach [0.3686808512438362]
  "What is said" through words (text) is more important than "how it is said" through imagery (video images) or acoustics (audio) in predicting video engagement.<n>We analyze unstructured data from long-form YouTube influencer videos.
  arXiv  Detail & Related papers  (2020-12-22T19:32:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.