Customized Visual Storytelling with Unified Multimodal LLMs

• URL: http://arxiv.org/abs/2603.27690v1
• Date: Sun, 29 Mar 2026 13:24:51 GMT
• Title: Customized Visual Storytelling with Unified Multimodal LLMs
• Abstract summary: VstoryGen is a framework that integrates descriptions with character and background references to enable customizable story generation.<n>We introduce shot-type control via parameter-efficient prompt tuning on movie data, enabling the model to generate sequences that more faithfully reflect cinematic grammar.
• Score: 19.106229466343787
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Multimodal story customization aims to generate coherent story flows conditioned on textual descriptions, reference identity images, and shot types. While recent progress in story generation has shown promising results, most approaches rely on text-only inputs. A few studies incorporate character identity cues (e.g., facial ID), but lack broader multimodal conditioning. In this work, we introduce VstoryGen, a multimodal framework that integrates descriptions with character and background references to enable customizable story generation. To enhance cinematic diversity, we introduce shot-type control via parameter-efficient prompt tuning on movie data, enabling the model to generate sequences that more faithfully reflect cinematic grammar. To evaluate our framework, we establish two new benchmarks that assess multimodal story customization from the perspectives of character and scene consistency, text-visual alignment, and shot-type control. Experiments demonstrate that VstoryGen achieves improved consistency and cinematic diversity compared to existing methods.

Related papers

• MAVIN: Multi-Shot Audio-Visual Generation with Narrative Control [66.04301887685004]
  Existing methods suffer from temporal misalignment, limited controllability, and incomplete scripting.<n>We propose MAVIN, the first framework for multi-shot audio-visual generation with customized narrative control.<n> MAVIN achieves state-of-the-art performance, opening up a new avenue for integrating generative models into professional filmmaking.
  arXiv  Detail & Related papers  (2026-06-28T16:01:04Z)
• Towards Customized Multimodal Role-Play [50.72454102691162]
  multimodal understanding and generation models enable richer human-AI interaction.<n>Yet jointly customizing a character's persona, dialogue style, and visual identity while maintaining output consistency across modalities remains largely unexplored.<n>We construct the RoleScape-20 dataset comprising 20 characters, including training and evaluation data that cover persona, stylistic descriptions, visual/expressive cues, and text-image interactions.<n>We devise UniCharacter, a two-stage training framework containing Unified Supervised Finetuning (Unified-SFT) and character-specific group relative policy optimization (Character-GRPO). Given only 10 images plus corresponding interaction examples, the model acquires the
  arXiv  Detail & Related papers  (2026-05-01T03:22:04Z)
• Persistent Story World Simulation with Continuous Character Customization [85.65195925369268]
  We present EverTale, a story world simulator for continuous story character customization.<n>Our EverTale achieves superior performance against a wider range of compared methods on both single- and multi-character story visualization.
  arXiv  Detail & Related papers  (2026-03-17T09:17:10Z)
• ViStoryBench: Comprehensive Benchmark Suite for Story Visualization [23.274981415638837]
  ViStoryBench is a comprehensive benchmark designed to evaluate story visualization models across diverse narrative structures, visual styles, and character settings.<n>The benchmark features richly annotated multi-shot scripts derived from curated stories spanning literature, film, and folklore.<n>To enable thorough evaluation, ViStoryBench introduces a set of automated metrics that assess character consistency, style similarity, prompt adherence, aesthetic quality, and generation artifacts.
  arXiv  Detail & Related papers  (2025-05-30T17:58:21Z)
• STORYANCHORS: Generating Consistent Multi-Scene Story Frames for Long-Form Narratives [82.19488717416351]
  This paper introduces StoryAnchors, a unified framework for generating high-quality, multi-scene story frames.<n>StoryAnchors employs a bidirectional story generator that integrates both past and future contexts to ensure temporal consistency.<n>It also integrates Multi-Event Story Frame Labeling and Progressive Story Frame Training, enabling the model to capture both overarching narrative flow and event-level dynamics.
  arXiv  Detail & Related papers  (2025-05-13T08:48:10Z)
• One-Prompt-One-Story: Free-Lunch Consistent Text-to-Image Generation Using a Single Prompt [101.17660804110409]
  Text-to-image generation models can create high-quality images from input prompts.<n>They struggle to support the consistent generation of identity-preserving requirements for storytelling.<n>We propose a novel training-free method for consistent text-to-image generation.
  arXiv  Detail & Related papers  (2025-01-23T10:57:22Z)
• Follow-Your-MultiPose: Tuning-Free Multi-Character Text-to-Video Generation via Pose Guidance [29.768141136041454]
  We propose a novel multi-character video generation framework, which is based on the separated text and pose guidance.<n>Specifically, we first extract character masks from the pose sequence to identify the spatial position for each generating character, and then single prompts for each character are obtained with LLMs.<n>The visualized results of generating video demonstrate the precise controllability of our method for multi-character generation.
  arXiv  Detail & Related papers  (2024-12-21T05:49:40Z)
• Storynizor: Consistent Story Generation via Inter-Frame Synchronized and Shuffled ID Injection [27.412361280397057]
  We introduce Storynizor, a model capable of generating coherent stories with strong inter-frame character consistency. Key innovation of Storynizor lies in its key modules: ID-Synchronizer and ID-Injector. To facilitate the training of Storynizor, we have curated a novel dataset called StoryDB comprising 100, 000 images.
  arXiv  Detail & Related papers  (2024-09-29T09:15:51Z)
• SEED-Story: Multimodal Long Story Generation with Large Language Model [66.37077224696242]
  SEED-Story is a novel method that leverages a Multimodal Large Language Model (MLLM) to generate extended multimodal stories. We propose multimodal attention sink mechanism to enable the generation of stories with up to 25 sequences (only 10 for training) in a highly efficient autoregressive manner. We present a large-scale and high-resolution dataset named StoryStream for training our model and quantitatively evaluating the task of multimodal story generation in various aspects.
  arXiv  Detail & Related papers  (2024-07-11T17:21:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.