Putting GPT-3's Creativity to the (Alternative Uses) Test
- URL: http://arxiv.org/abs/2206.08932v1
- Date: Fri, 10 Jun 2022 15:36:45 GMT
- Title: Putting GPT-3's Creativity to the (Alternative Uses) Test
- Authors: Claire Stevenson, Iris Smal, Matthijs Baas, Raoul Grasman and Han van
der Maas
- Abstract summary: We put Open AI's generative natural language model, GPT-3, to the test.
We assessed GPT-3's creativity on Guilford's Alternative Uses Test.
Humans currently outperform GPT-3 when it comes to creative output.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: AI large language models have (co-)produced amazing written works from
newspaper articles to novels and poetry. These works meet the standards of the
standard definition of creativity: being original and useful, and sometimes
even the additional element of surprise. But can a large language model
designed to predict the next text fragment provide creative, out-of-the-box,
responses that still solve the problem at hand? We put Open AI's generative
natural language model, GPT-3, to the test. Can it provide creative solutions
to one of the most commonly used tests in creativity research? We assessed
GPT-3's creativity on Guilford's Alternative Uses Test and compared its
performance to previously collected human responses on expert ratings of
originality, usefulness and surprise of responses, flexibility of each set of
ideas as well as an automated method to measure creativity based on the
semantic distance between a response and the AUT object in question. Our
results show that -- on the whole -- humans currently outperform GPT-3 when it
comes to creative output. But, we believe it is only a matter of time before
GPT-3 catches up on this particular task. We discuss what this work reveals
about human and AI creativity, creativity testing and our definition of
creativity.
Related papers
- Steering Large Language Models to Evaluate and Amplify Creativity [7.031631627161492]
We show that we can leverage this knowledge of how to write creatively in order to better judge what is creative.
We take a mechanistic approach that extracts differences in the internal states of an LLM when prompted to respond "boringly" or "creatively"
arXiv Detail & Related papers (2024-12-08T20:28:48Z) - "It was 80% me, 20% AI": Seeking Authenticity in Co-Writing with Large Language Models [97.22914355737676]
We examine whether and how writers want to preserve their authentic voice when co-writing with AI tools.
Our findings illuminate conceptions of authenticity in human-AI co-creation.
Readers' responses showed less concern about human-AI co-writing.
arXiv Detail & Related papers (2024-11-20T04:42:32Z) - AI as Humanity's Salieri: Quantifying Linguistic Creativity of Language Models via Systematic Attribution of Machine Text against Web Text [53.15652021126663]
We present CREATIVITY INDEX as the first step to quantify the linguistic creativity of a text.
To compute CREATIVITY INDEX efficiently, we introduce DJ SEARCH, a novel dynamic programming algorithm.
Experiments reveal that the CREATIVITY INDEX of professional human authors is on average 66.2% higher than that of LLMs.
arXiv Detail & Related papers (2024-10-05T18:55:01Z) - Can AI Enhance its Creativity to Beat Humans ? [0.0]
This study investigates the creative performance of artificial intelligence (AI) compared to humans.
Human external evaluators have scored creative outputs generated by humans and AI.
Results suggest that integrating human feedback is crucial for maximizing AI's creative potential.
arXiv Detail & Related papers (2024-09-27T14:19:07Z) - Investigating Wit, Creativity, and Detectability of Large Language Models in Domain-Specific Writing Style Adaptation of Reddit's Showerthoughts [17.369951848952265]
We investigate the ability of LLMs to replicate human writing style in short, creative texts in the domain of Showerthoughts.
We measure human preference on the texts across the specific dimensions that account for the quality of creative, witty texts.
We conclude that human evaluators rate the generated texts slightly worse on average regarding their creative quality, but they are unable to reliably distinguish between human-written and AI-generated texts.
arXiv Detail & Related papers (2024-05-02T18:29:58Z) - Can AI Be as Creative as Humans? [84.43873277557852]
We prove in theory that AI can be as creative as humans under the condition that it can properly fit the data generated by human creators.
The debate on AI's creativity is reduced into the question of its ability to fit a sufficient amount of data.
arXiv Detail & Related papers (2024-01-03T08:49:12Z) - Art or Artifice? Large Language Models and the False Promise of
Creativity [53.04834589006685]
We propose the Torrance Test of Creative Writing (TTCW) to evaluate creativity as a product.
TTCW consists of 14 binary tests organized into the original dimensions of Fluency, Flexibility, Originality, and Elaboration.
Our analysis shows that LLM-generated stories pass 3-10X less TTCW tests than stories written by professionals.
arXiv Detail & Related papers (2023-09-25T22:02:46Z) - AI, write an essay for me: A large-scale comparison of human-written
versus ChatGPT-generated essays [66.36541161082856]
ChatGPT and similar generative AI models have attracted hundreds of millions of users.
This study compares human-written versus ChatGPT-generated argumentative student essays.
arXiv Detail & Related papers (2023-04-24T12:58:28Z) - Artificial muses: Generative Artificial Intelligence Chatbots Have Risen
to Human-Level Creativity [1.332560004325655]
We compare human-generated ideas with those generated by six Generative Artificial Intelligence (GAI)
We found no qualitative difference between AI and human-generated creativity, although there are differences in how ideas are generated.
Our findings suggest that GAIs are valuable assistants in the creative process.
arXiv Detail & Related papers (2023-03-21T16:35:01Z) - DeepCreativity: Measuring Creativity with Deep Learning Techniques [2.5426469613007012]
This paper explores the possibility of using generative learning techniques for automatic assessment of creativity.
We introduce a new measure, namely DeepCreativity, based on Margaret Boden's definition of creativity as composed by value, novelty and surprise.
arXiv Detail & Related papers (2022-01-16T19:00:01Z) - Telling Creative Stories Using Generative Visual Aids [52.623545341588304]
We asked writers to write creative stories from a starting prompt, and provided them with visuals created by generative AI models from the same prompt.
Compared to a control group, writers who used the visuals as story writing aid wrote significantly more creative, original, complete and visualizable stories.
Findings indicate that cross modality inputs by AI can benefit divergent aspects of creativity in human-AI co-creation, but hinders convergent thinking.
arXiv Detail & Related papers (2021-10-27T23:13:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.