EmoStory: Emotion-Aware Story Generation

• URL: http://arxiv.org/abs/2603.10349v2
• Date: Tue, 17 Mar 2026 04:56:27 GMT
• Title: EmoStory: Emotion-Aware Story Generation
• Abstract summary: We introduce emotion-aware story generation, a new task that aims to generate subject-consistent visual stories with explicit emotional directions.<n>EmoStory is a two-stage framework that integrates agent-based story planning and region-aware story generation.<n>We evaluate EmoStory on a newly constructed dataset covering 25 subjects and 600 emotional stories.
• Score: 8.859038452064818
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Story generation aims to produce image sequences that depict coherent narratives while maintaining subject consistency across frames. Although existing methods have excelled in producing coherent and expressive stories, they remain largely emotion-neutral, focusing on what subject appears in a story while overlooking how emotions shape narrative interpretation and visual presentation. As stories are intended to engage audiences emotionally, we introduce emotion-aware story generation, a new task that aims to generate subject-consistent visual stories with explicit emotional directions. This task is challenging due to the abstract nature of emotions, which must be grounded in concrete visual elements and consistently expressed across a narrative through visual composition. To address these challenges, we propose EmoStory, a two-stage framework that integrates agent-based story planning and region-aware story generation. The planning stage transforms target emotions into coherent story prompts with emotion agent and writer agent, while the generation stage preserves subject consistency and injects emotion-related elements through region-aware composition. We evaluate EmoStory on a newly constructed dataset covering 25 subjects and 600 emotional stories. Extensive quantitative and qualitative results, along with user studies, show that EmoStory outperforms state-of-the-art story generation methods in emotion accuracy, prompt alignment, and subject consistency.

Related papers

• Steering Emotional Dynamics for Art Therapy: Controllable Narrative Script Generation through Hierarchically Guided LLM Agents [15.997100974367195]
  Art therapy plays a vital role in emotional healing, in which narrative creation acts as the primary vehicle for emotional expression.<n>With the rapid development of Large Language Models (LLMs), automated narrative generation technology has provided a new pathway to support such artistic designs.<n>This paper proposes EC-Script, an agent-based framework that enables hierarchical control of the affective trajectory in narrative generation for emotional healing.
  arXiv  Detail & Related papers  (2026-06-15T09:49:23Z)
• Text-Driven Emotionally Continuous Talking Face Generation [24.062908127306695]
  We propose Emotionally Continuous Talking Face Generation (EC-TFG) to synthesize realistic videos.<n>EC-TFG takes a text segment and an emotion description with varying emotions as driving data, aiming to generate a video where the person speaks the text.<n>We introduce a customized model, i.e., Temporal-Intensive Emotion Modulated Talking Face Generation (TIE-TFG), which innovatively manages dynamic emotional variations.
  arXiv  Detail & Related papers  (2026-03-06T09:24:43Z)
• Taming Transformer for Emotion-Controllable Talking Face Generation [61.835295250047196]
  We propose a novel method to tackle the emotion-controllable talking face generation task discretely.<n>Specifically, we employ two pre-training strategies to disentangle audio into independent components and quantize videos into combinations of visual tokens.<n>We conduct experiments on the MEAD dataset that controls the emotion of videos conditioned on multiple emotional audios.
  arXiv  Detail & Related papers  (2025-08-20T02:16:52Z)
• All Stories Are One Story: Emotional Arc Guided Procedural Game Level Generation [1.8885685625700497]
  We present a framework for procedural game narrative generation that incorporates emotional arcs as a structural backbone.<n>We focus on two core emotional patterns -- Rise and Fall -- to guide the generation of branching story graphs.<n>Our system demonstrates how emotional arcs can be operationalized using large language models and adaptive entity generation.
  arXiv  Detail & Related papers  (2025-08-04T07:27:55Z)
• Disentangle Identity, Cooperate Emotion: Correlation-Aware Emotional Talking Portrait Generation [63.94836524433559]
  DICE-Talk is a framework for disentangling identity with emotion and cooperating emotions with similar characteristics.<n>We develop a disentangled emotion embedder that jointly models audio-visual emotional cues through cross-modal attention.<n>Second, we introduce a correlation-enhanced emotion conditioning module with learnable Emotion Banks.<n>Third, we design an emotion discrimination objective that enforces affective consistency during the diffusion process.
  arXiv  Detail & Related papers  (2025-04-25T05:28:21Z)
• Dual-path Collaborative Generation Network for Emotional Video Captioning [33.230028098522254]
  Emotional Video Captioning is an emerging task that aims to describe factual content with the intrinsic emotions expressed in videos. Existing emotional video captioning methods perceive global visual emotional cues at first, and then combine them with the video features to guide the emotional caption generation. We propose a dual-path collaborative generation network, which dynamically perceives visual emotional cues evolutions while generating emotional captions.
  arXiv  Detail & Related papers  (2024-08-06T07:30:53Z)
• EmoTalker: Emotionally Editable Talking Face Generation via Diffusion Model [39.14430238946951]
  EmoTalker is an emotionally editable portraits animation approach based on the diffusion model. Emotion Intensity Block is introduced to analyze fine-grained emotions and strengths derived from prompts. Experiments show the effectiveness of EmoTalker in generating high-quality, emotionally customizable facial expressions.
  arXiv  Detail & Related papers  (2024-01-16T02:02:44Z)
• Emotion Rendering for Conversational Speech Synthesis with Heterogeneous Graph-Based Context Modeling [50.99252242917458]
  Conversational Speech Synthesis (CSS) aims to accurately express an utterance with the appropriate prosody and emotional inflection within a conversational setting. To address the issue of data scarcity, we meticulously create emotional labels in terms of category and intensity. Our model outperforms the baseline models in understanding and rendering emotions.
  arXiv  Detail & Related papers  (2023-12-19T08:47:50Z)
• Speech Synthesis with Mixed Emotions [77.05097999561298]
  We propose a novel formulation that measures the relative difference between the speech samples of different emotions. We then incorporate our formulation into a sequence-to-sequence emotional text-to-speech framework. At run-time, we control the model to produce the desired emotion mixture by manually defining an emotion attribute vector.
  arXiv  Detail & Related papers  (2022-08-11T15:45:58Z)
• ViNTER: Image Narrative Generation with Emotion-Arc-Aware Transformer [59.05857591535986]
  We propose a model called ViNTER to generate image narratives that focus on time series representing varying emotions as "emotion arcs" We present experimental results of both manual and automatic evaluations.
  arXiv  Detail & Related papers  (2022-02-15T10:53:08Z)
• SOLVER: Scene-Object Interrelated Visual Emotion Reasoning Network [83.27291945217424]
  We propose a novel Scene-Object interreLated Visual Emotion Reasoning network (SOLVER) to predict emotions from images. To mine the emotional relationships between distinct objects, we first build up an Emotion Graph based on semantic concepts and visual features. We also design a Scene-Object Fusion Module to integrate scenes and objects, which exploits scene features to guide the fusion process of object features with the proposed scene-based attention mechanism.
  arXiv  Detail & Related papers  (2021-10-24T02:41:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.