Coherence-Oriented Dream Scene Visualisation

• URL: http://arxiv.org/abs/2608.05233v1
• Date: Wed, 05 Aug 2026 13:36:34 GMT
• Title: Coherence-Oriented Dream Scene Visualisation
• Abstract summary: We describe a system which turns written dream descriptions into a temporal sequence of four panel images visualising the dream.<n>This starts with a large language model prompted to split a dream description into four chronological parts.<n>We evaluate DSV over 50 visualisations from dream descriptions in DreamBank, and report quality, fidelity and coherence results.
• Score: 3.7323123012231285
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dreams can be emotionally intense but difficult to communicate. We describe the Dream Scene Visualiser (DSV) system which turns written dream descriptions into a temporal sequence of four panel images visualising the dream. This starts with a large language model prompted to split a dream description into four chronological parts. Then a text-to-image model produces images for each part with visual coherence maintained across the sequence, and DSV regenerates any image not suitably matching the text. We evaluate DSV over 50 visualisations from dream descriptions in DreamBank, and report quality, fidelity and coherence results via objective measures employing the CLIP, DINOv2 and Qwen2-VL vision-language models.

Related papers

• Dreaming in Flow: Generative Grounding Feedback for Self-Evolving Unified Multimodal Models [55.231245660246934]
  Unified multimodal models integrate visual understanding and generation within a single network, yet the two capabilities are commonly optimized as separate tasks.<n>We introduce Generative Grounding Feedback, a self-evolving post-training framework that uses only text prompts and the model's own visual experience.
  arXiv  Detail & Related papers  (2026-09-08T05:52:53Z)
• DreamShot: Personalized Storyboard Synthesis with Video Diffusion Prior [84.56343116275423]
  We introduce DreamShot, a video generative model based storyboard framework.<n>DreamShot exploits powerful video diffusion priors for controllable multi-shot synthesis.<n>It produces visually and semantically coherent sequences with improved narrative fidelity and character continuity.
  arXiv  Detail & Related papers  (2026-04-19T01:51:41Z)
• Making Your Dreams A Reality: Decoding the Dreams into a Coherent Video Story from fMRI Signals [46.90535445975669]
  This paper studies the brave new idea for Multimedia community, and proposes a novel framework to convert dreams into coherent video narratives.<n>Recent advancements in brain imaging, particularly functional magnetic resonance imaging (fMRI), have provided new ways to explore the neural basis of dreaming.<n>By combining subjective dream experiences with objective neurophysiological data, we aim to understand the visual aspects of dreams and create complete video narratives.
  arXiv  Detail & Related papers  (2025-01-16T08:03:49Z)
• DreamRunner: Fine-Grained Compositional Story-to-Video Generation with Retrieval-Augmented Motion Adaptation [60.07447565026327]
  We propose DreamRunner, a novel story-to-video generation method.<n>We structure the input script using a large language model (LLM) to facilitate both coarse-grained scene planning and fine-grained object-level layout and motion planning.<n>DreamRunner presents retrieval-augmented test-time adaptation to capture target motion priors for objects in each scene, supporting diverse motion customization based on retrieved videos.
  arXiv  Detail & Related papers  (2024-11-25T18:41:56Z)
• DreamStory: Open-Domain Story Visualization by LLM-Guided Multi-Subject Consistent Diffusion [28.61450171549731]
  We propose DreamStory, an automatic open-domain story visualization framework.<n>DreamStory consists of (1) an LLM acting as a story director and (2) an innovative Multi-Subject consistent Diffusion model (MSD)<n>The MSD includes Masked Mutual Self-Attention (MMSA) and Masked Mutual Cross-Attention (MMCA) modules.
  arXiv  Detail & Related papers  (2024-07-17T17:54:12Z)
• Sequence-to-Sequence Language Models for Character and Emotion Detection in Dream Narratives [0.0]
  This paper presents the first study on character and emotion detection in the English portion of the open DreamBank corpus of dream narratives. Our results show that language models can effectively address this complex task. We evaluate the impact of model size, prediction order of characters, and the consideration of proper names and character traits.
  arXiv  Detail & Related papers  (2024-03-21T08:27:49Z)
• Visual Storytelling with Question-Answer Plans [70.89011289754863]
  We present a novel framework which integrates visual representations with pretrained language models and planning. Our model translates the image sequence into a visual prefix, a sequence of continuous embeddings which language models can interpret. It also leverages a sequence of question-answer pairs as a blueprint plan for selecting salient visual concepts and determining how they should be assembled into a narrative.
  arXiv  Detail & Related papers  (2023-10-08T21:45:34Z)
• DeViL: Decoding Vision features into Language [53.88202366696955]
  Post-hoc explanation methods have often been criticised for abstracting away the decision-making process of deep neural networks. In this work, we would like to provide natural language descriptions for what different layers of a vision backbone have learned. We train a transformer network to translate individual image features of any vision layer into a prompt that a separate off-the-shelf language model decodes into natural language.
  arXiv  Detail & Related papers  (2023-09-04T13:59:55Z)
• DreamingV2: Reinforcement Learning with Discrete World Models without Reconstruction [14.950054143767824]
  The present paper proposes a novel reinforcement learning method with world models, DreamingV2. DreamingV2 is a collaborative extension of DreamerV2 and Dreaming. We believe DreamingV2 will be a reliable solution for robot learning since its discrete representation is suitable to describe discontinuous environments.
  arXiv  Detail & Related papers  (2022-03-01T14:44:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.