Telling Creative Stories Using Generative Visual Aids

• URL: http://arxiv.org/abs/2110.14810v1
• Date: Wed, 27 Oct 2021 23:13:47 GMT
• Title: Telling Creative Stories Using Generative Visual Aids
• Authors: Safinah Ali, Devi Parikh
• Abstract summary: We asked writers to write creative stories from a starting prompt, and provided them with visuals created by generative AI models from the same prompt. Compared to a control group, writers who used the visuals as story writing aid wrote significantly more creative, original, complete and visualizable stories. Findings indicate that cross modality inputs by AI can benefit divergent aspects of creativity in human-AI co-creation, but hinders convergent thinking.
• Score: 52.623545341588304
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Can visual artworks created using generative visual algorithms inspire human creativity in storytelling? We asked writers to write creative stories from a starting prompt, and provided them with visuals created by generative AI models from the same prompt. Compared to a control group, writers who used the visuals as story writing aid wrote significantly more creative, original, complete and visualizable stories, and found the task more fun. Of the generative algorithms used (BigGAN, VQGAN, DALL-E, CLIPDraw), VQGAN was the most preferred. The control group that did not view the visuals did significantly better in integrating the starting prompts. Findings indicate that cross modality inputs by AI can benefit divergent aspects of creativity in human-AI co-creation, but hinders convergent thinking.

Related papers

• A Character-Centric Creative Story Generation via Imagination [15.345466372805516]
  We introduce a novel story generation framework called CCI (Character-centric Creative story generation via Imagination) CCI features two modules for creative story generation: IG (Image-Guided Imagination) and MW (Multi-Writer model) In the IG module, we utilize a text-to-image model to create visual representations of key story elements, such as characters, backgrounds, and main plots. The MW module uses these story elements to generate multiple persona-description candidates and selects the best one to insert into the story, thereby enhancing the richness and depth of the narrative.
  arXiv  Detail & Related papers  (2024-09-25T06:54:29Z)
• SARD: A Human-AI Collaborative Story Generation [0.0]
  We propose SARD, a drag-and-drop visual interface for generating a multi-chapter story using large language models. Our evaluation of the usability of SARD and its creativity support shows that while node-based visualization of the narrative may help writers build a mental model, it exerts unnecessary mental overhead to the writer. We also found that AI generates stories that are less lexically diverse, irrespective of the complexity of the story.
  arXiv  Detail & Related papers  (2024-03-03T17:48:42Z)
• MagicScroll: Nontypical Aspect-Ratio Image Generation for Visual Storytelling via Multi-Layered Semantic-Aware Denoising [42.20750912837316]
  MagicScroll is a progressive diffusion-based image generation framework with a novel semantic-aware denoising process. It enables fine-grained control over the generated image on object, scene, and background levels with text, image, and layout conditions. It showcases promising results in aligning with the narrative text, improving visual coherence, and engaging the audience.
  arXiv  Detail & Related papers  (2023-12-18T03:09:05Z)
• Text-Only Training for Visual Storytelling [107.19873669536523]
  We formulate visual storytelling as a visual-conditioned story generation problem. We propose a text-only training method that separates the learning of cross-modality alignment and story generation.
  arXiv  Detail & Related papers  (2023-08-17T09:32:17Z)
• Intelligent Grimm -- Open-ended Visual Storytelling via Latent Diffusion Models [70.86603627188519]
  We focus on a novel, yet challenging task of generating a coherent image sequence based on a given storyline, denoted as open-ended visual storytelling. We propose a learning-based auto-regressive image generation model, termed as StoryGen, with a novel vision-language context module. We show StoryGen can generalize to unseen characters without any optimization, and generate image sequences with coherent content and consistent character.
  arXiv  Detail & Related papers  (2023-06-01T17:58:50Z)
• Visualize Before You Write: Imagination-Guided Open-Ended Text Generation [68.96699389728964]
  We propose iNLG that uses machine-generated images to guide language models in open-ended text generation. Experiments and analyses demonstrate the effectiveness of iNLG on open-ended text generation tasks.
  arXiv  Detail & Related papers  (2022-10-07T18:01:09Z)
• Creative Wand: A System to Study Effects of Communications in Co-Creative Settings [9.356870107137093]
  Co-creative, mixed-initiative systems require user-centric means of influencing the algorithm. Key questions in co-creative AI include: How can users express their creative intentions? We introduce CREATIVE-WAND, a customizable framework for investigating co-creative mixed-initiative generation.
  arXiv  Detail & Related papers  (2022-08-04T20:56:40Z)
• Towards Coherent Visual Storytelling with Ordered Image Attention [73.422281039592]
  We develop ordered image attention (OIA) and Image-Sentence Attention (ISA) OIA models interactions between the sentence-corresponding image and important regions in other images of the sequence. To generate the story's sentences, we then highlight important image attention vectors with an Image-Sentence Attention (ISA)
  arXiv  Detail & Related papers  (2021-08-04T17:12:39Z)
• FairyTailor: A Multimodal Generative Framework for Storytelling [33.39639788612019]
  We introduce a system and a demo, FairyTailor, for human-in-the-loop visual story co-creation. Users can create a cohesive children's fairytale by weaving generated texts and retrieved images with their input. To our knowledge, this is the first dynamic tool for multimodal story generation that allows interactive co-formation of both texts and images.
  arXiv  Detail & Related papers  (2021-07-13T02:45:08Z)
• Hide-and-Tell: Learning to Bridge Photo Streams for Visual Storytelling [86.42719129731907]
  We propose to explicitly learn to imagine a storyline that bridges the visual gap. We train the network to produce a full plausible story even with missing photo(s) In experiments, we show that our scheme of hide-and-tell, and the network design are indeed effective at storytelling.
  arXiv  Detail & Related papers  (2020-02-03T14:22:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.