Experimental Narratives: A Comparison of Human Crowdsourced Storytelling
and AI Storytelling
- URL: http://arxiv.org/abs/2310.12902v1
- Date: Thu, 19 Oct 2023 16:54:38 GMT
- Title: Experimental Narratives: A Comparison of Human Crowdsourced Storytelling
and AI Storytelling
- Authors: Nina Begus
- Abstract summary: The study analyzes 250 stories authored by crowdworkers in June 2019 and 80 stories generated by GPT-3.5 and GPT-4.
Both crowdworkers and large language models responded to identical prompts about creating and falling in love with an artificial human.
The analysis reveals that narratives from GPT-3.5 and particularly GPT-4 are more progressive in terms of gender roles and sexuality than those written by humans.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The paper proposes a framework that combines behavioral and computational
experiments employing fictional prompts as a novel tool for investigating
cultural artifacts and social biases in storytelling both by humans and
generative AI. The study analyzes 250 stories authored by crowdworkers in June
2019 and 80 stories generated by GPT-3.5 and GPT-4 in March 2023 by merging
methods from narratology and inferential statistics. Both crowdworkers and
large language models responded to identical prompts about creating and falling
in love with an artificial human. The proposed experimental paradigm allows a
direct comparison between human and LLM-generated storytelling. Responses to
the Pygmalionesque prompts confirm the pervasive presence of the Pygmalion myth
in the collective imaginary of both humans and large language models. All
solicited narratives present a scientific or technological pursuit. The
analysis reveals that narratives from GPT-3.5 and particularly GPT-4 are more
more progressive in terms of gender roles and sexuality than those written by
humans. While AI narratives can occasionally provide innovative plot twists,
they offer less imaginative scenarios and rhetoric than human-authored texts.
The proposed framework argues that fiction can be used as a window into human
and AI-based collective imaginary and social dimensions.
Related papers
- Are Large Language Models Capable of Generating Human-Level Narratives? [114.34140090869175]
This paper investigates the capability of LLMs in storytelling, focusing on narrative development and plot progression.
We introduce a novel computational framework to analyze narratives through three discourse-level aspects.
We show that explicit integration of discourse features can enhance storytelling, as is demonstrated by over 40% improvement in neural storytelling.
arXiv Detail & Related papers (2024-07-18T08:02:49Z) - The GPT-WritingPrompts Dataset: A Comparative Analysis of Character Portrayal in Short Stories [17.184517720465404]
We quantify and compare the emotional and descriptive features of storytelling from both generative processes, human and machine, along a set of six dimensions.
We find that generated stories differ significantly from human stories along all six dimensions, and that human and machine generations display similar biases when grouped according to the narrative point-of-view and gender of the main protagonist.
arXiv Detail & Related papers (2024-06-24T16:24:18Z) - Measuring Psychological Depth in Language Models [48.94536312831725]
We introduce the Psychological Depth Scale (PDS), a novel framework rooted in literary theory that measures an LLM's ability to produce authentic and narratively complex stories.
We empirically validate our framework by showing that humans can consistently evaluate stories based on PDS (0.72 Krippendorff's alpha)
Surprisingly, GPT-4 stories either surpassed or were statistically indistinguishable from highly-rated human-written stories sourced from Reddit.
arXiv Detail & Related papers (2024-06-18T14:51:54Z) - Creating Suspenseful Stories: Iterative Planning with Large Language
Models [2.6923151107804055]
We propose a novel iterative-prompting-based planning method that is grounded in two theoretical foundations of story suspense.
To the best of our knowledge, this paper is the first attempt at suspenseful story generation with large language models.
arXiv Detail & Related papers (2024-02-27T01:25:52Z) - Affective and Dynamic Beam Search for Story Generation [50.3130767805383]
We propose Affective Story Generator (AffGen) for generating interesting narratives.
AffGen employs two novel techniques-Dynamic Beam Sizing and Affective Reranking.
arXiv Detail & Related papers (2023-10-23T16:37:14Z) - Computational Storytelling and Emotions: A Survey [56.95572957863576]
This survey paper is intended to summarize and contribute to the development of research being conducted on the relationship between stories and emotions.
We believe creativity research is not to replace humans with computers, but to find a way of collaboration between humans and computers to enhance the creativity.
arXiv Detail & Related papers (2022-05-23T00:21:59Z) - Persona-Guided Planning for Controlling the Protagonist's Persona in
Story Generation [71.24817035071176]
We propose a planning-based generation model named CONPER to explicitly model the relationship between personas and events.
Both automatic and manual evaluation results demonstrate that CONPER outperforms state-of-the-art baselines for generating more coherent and persona-controllable stories.
arXiv Detail & Related papers (2022-04-22T13:45:02Z) - Computational Lens on Cognition: Study Of Autobiographical Versus
Imagined Stories With Large-Scale Language Models [95.88620740809004]
We study differences in the narrative flow of events in autobiographical versus imagined stories using GPT-3.
We found that imagined stories have higher sequentiality than autobiographical stories.
In comparison to imagined stories, autobiographical stories contain more concrete words and words related to the first person.
arXiv Detail & Related papers (2022-01-07T20:10:47Z) - Collaborative Storytelling with Human Actors and AI Narrators [2.8575516056239576]
We report on using GPT-3 citebrown 2020 to co-narrate stories.
The AI system must track plot progression and character arcs while the human actors perform scenes.
arXiv Detail & Related papers (2021-09-29T21:21:35Z) - Exploring aspects of similarity between spoken personal narratives by
disentangling them into narrative clause types [13.350982138577038]
We introduce a corpus of real-world spoken personal narratives comprising 10,296 narrative clauses from 594 video transcripts.
Second, we ask non-narrative experts to annotate those clauses under Labov's sociolinguistic model of personal narratives.
Third, we train a classifier that reaches 84.7% F-score for the highest-agreed clauses.
Our approach is intended to help inform machine learning methods aimed at studying or representing personal narratives.
arXiv Detail & Related papers (2020-05-26T14:34:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.