Envisioning Narrative Intelligence: A Creative Visual Storytelling
Anthology
- URL: http://arxiv.org/abs/2310.04529v1
- Date: Fri, 6 Oct 2023 18:47:20 GMT
- Title: Envisioning Narrative Intelligence: A Creative Visual Storytelling
Anthology
- Authors: Brett A. Halperin and Stephanie M. Lukin
- Abstract summary: We present five themes that characterize the variations found in this creative visual storytelling process.
We envision narrative intelligence criteria for computational visual storytelling as: creative, reliable, expressive, grounded, and responsible.
- Score: 7.962160810367763
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In this paper, we collect an anthology of 100 visual stories from authors who
participated in our systematic creative process of improvised story-building
based on image sequences. Following close reading and thematic analysis of our
anthology, we present five themes that characterize the variations found in
this creative visual storytelling process: (1) Narrating What is in Vision vs.
Envisioning; (2) Dynamically Characterizing Entities/Objects; (3) Sensing
Experiential Information About the Scenery; (4) Modulating the Mood; (5)
Encoding Narrative Biases. In understanding the varied ways that people derive
stories from images, we offer considerations for collecting story-driven
training data to inform automatic story generation. In correspondence with each
theme, we envision narrative intelligence criteria for computational visual
storytelling as: creative, reliable, expressive, grounded, and responsible.
From these criteria, we discuss how to foreground creative expression, account
for biases, and operate in the bounds of visual storyworlds.
Related papers
- Generating Visual Stories with Grounded and Coreferent Characters [63.07511918366848]
We present the first model capable of predicting visual stories with consistently grounded and coreferent character mentions.
Our model is finetuned on a new dataset which we build on top of the widely used VIST benchmark.
We also propose new evaluation metrics to measure the richness of characters and coreference in stories.
arXiv Detail & Related papers (2024-09-20T14:56:33Z) - TARN-VIST: Topic Aware Reinforcement Network for Visual Storytelling [14.15543866199545]
As a cross-modal task, visual storytelling aims to generate a story for an ordered image sequence automatically.
We propose a novel method, Topic Aware Reinforcement Network for VIsual StoryTelling (TARN-VIST)
In particular, we pre-extracted the topic information of stories from both visual and linguistic perspectives.
arXiv Detail & Related papers (2024-03-18T08:01:23Z) - SCO-VIST: Social Interaction Commonsense Knowledge-based Visual
Storytelling [12.560014305032437]
This paper introduces SCO-VIST, a framework representing the image sequence as a graph with objects and relations.
SCO-VIST then takes this graph representing plot points and creates bridges between plot points with semantic and occurrence-based edge weights.
This weighted story graph produces the storyline in a sequence of events using Floyd-Warshall's algorithm.
arXiv Detail & Related papers (2024-02-01T04:09:17Z) - Visual Storytelling with Question-Answer Plans [70.89011289754863]
We present a novel framework which integrates visual representations with pretrained language models and planning.
Our model translates the image sequence into a visual prefix, a sequence of continuous embeddings which language models can interpret.
It also leverages a sequence of question-answer pairs as a blueprint plan for selecting salient visual concepts and determining how they should be assembled into a narrative.
arXiv Detail & Related papers (2023-10-08T21:45:34Z) - ViNTER: Image Narrative Generation with Emotion-Arc-Aware Transformer [59.05857591535986]
We propose a model called ViNTER to generate image narratives that focus on time series representing varying emotions as "emotion arcs"
We present experimental results of both manual and automatic evaluations.
arXiv Detail & Related papers (2022-02-15T10:53:08Z) - Telling Creative Stories Using Generative Visual Aids [52.623545341588304]
We asked writers to write creative stories from a starting prompt, and provided them with visuals created by generative AI models from the same prompt.
Compared to a control group, writers who used the visuals as story writing aid wrote significantly more creative, original, complete and visualizable stories.
Findings indicate that cross modality inputs by AI can benefit divergent aspects of creativity in human-AI co-creation, but hinders convergent thinking.
arXiv Detail & Related papers (2021-10-27T23:13:47Z) - A guided journey through non-interactive automatic story generation [0.0]
The article presents requirements for creative systems, three types of models of creativity (computational, socio-cultural, and individual), and models of human creative writing.
The article concludes that the autonomous generation and adoption of the main idea to be conveyed and the autonomous design of the creativity ensuring criteria are possibly two of most important topics for future research.
arXiv Detail & Related papers (2021-10-08T10:01:36Z) - PlotMachines: Outline-Conditioned Generation with Dynamic Plot State
Tracking [128.76063992147016]
We present PlotMachines, a neural narrative model that learns to transform an outline into a coherent story by tracking the dynamic plot states.
In addition, we enrich PlotMachines with high-level discourse structure so that the model can learn different writing styles corresponding to different parts of the narrative.
arXiv Detail & Related papers (2020-04-30T17:16:31Z) - Hide-and-Tell: Learning to Bridge Photo Streams for Visual Storytelling [86.42719129731907]
We propose to explicitly learn to imagine a storyline that bridges the visual gap.
We train the network to produce a full plausible story even with missing photo(s)
In experiments, we show that our scheme of hide-and-tell, and the network design are indeed effective at storytelling.
arXiv Detail & Related papers (2020-02-03T14:22:18Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.