Putting GPT-3's Creativity to the (Alternative Uses) Test
- URL: http://arxiv.org/abs/2206.08932v1
- Date: Fri, 10 Jun 2022 15:36:45 GMT
- Title: Putting GPT-3's Creativity to the (Alternative Uses) Test
- Authors: Claire Stevenson, Iris Smal, Matthijs Baas, Raoul Grasman and Han van
der Maas
- Abstract summary: We put Open AI's generative natural language model, GPT-3, to the test.
We assessed GPT-3's creativity on Guilford's Alternative Uses Test.
Humans currently outperform GPT-3 when it comes to creative output.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: AI large language models have (co-)produced amazing written works from
newspaper articles to novels and poetry. These works meet the standards of the
standard definition of creativity: being original and useful, and sometimes
even the additional element of surprise. But can a large language model
designed to predict the next text fragment provide creative, out-of-the-box,
responses that still solve the problem at hand? We put Open AI's generative
natural language model, GPT-3, to the test. Can it provide creative solutions
to one of the most commonly used tests in creativity research? We assessed
GPT-3's creativity on Guilford's Alternative Uses Test and compared its
performance to previously collected human responses on expert ratings of
originality, usefulness and surprise of responses, flexibility of each set of
ideas as well as an automated method to measure creativity based on the
semantic distance between a response and the AUT object in question. Our
results show that -- on the whole -- humans currently outperform GPT-3 when it
comes to creative output. But, we believe it is only a matter of time before
GPT-3 catches up on this particular task. We discuss what this work reveals
about human and AI creativity, creativity testing and our definition of
creativity.
Related papers
- Automating Creativity [1.0200170217746136]
This paper explores what is required to evolve AI from generative to creative.
We develop a triple prompt-response-reward engineering framework to develop the creative capability of GenAI.
arXiv Detail & Related papers (2024-05-11T05:05:10Z) - Investigating Wit, Creativity, and Detectability of Large Language Models in Domain-Specific Writing Style Adaptation of Reddit's Showerthoughts [17.369951848952265]
We investigate the ability of LLMs to replicate human writing style in short, creative texts in the domain of Showerthoughts.
We measure human preference on the texts across the specific dimensions that account for the quality of creative, witty texts.
We conclude that human evaluators rate the generated texts slightly worse on average regarding their creative quality, but they are unable to reliably distinguish between human-written and AI-generated texts.
arXiv Detail & Related papers (2024-05-02T18:29:58Z) - Can AI Outperform Human Experts in Creating Social Media Creatives? [0.6963971634605797]
This paper evaluates AI's capability in the creative domain compared to human experts.
We propose a novel Prompt-for-Prompt to generate social media creatives via prompt augmentation by Large Language Models.
We find that AI excels human experts, and Midjourney is better than the other text-to-image generators.
arXiv Detail & Related papers (2024-03-19T07:41:45Z) - Can AI Be as Creative as Humans? [84.43873277557852]
We prove in theory that AI can be as creative as humans under the condition that it can properly fit the data generated by human creators.
The debate on AI's creativity is reduced into the question of its ability to fit a sufficient amount of data.
arXiv Detail & Related papers (2024-01-03T08:49:12Z) - Art or Artifice? Large Language Models and the False Promise of
Creativity [53.04834589006685]
We propose the Torrance Test of Creative Writing (TTCW) to evaluate creativity as a product.
TTCW consists of 14 binary tests organized into the original dimensions of Fluency, Flexibility, Originality, and Elaboration.
Our analysis shows that LLM-generated stories pass 3-10X less TTCW tests than stories written by professionals.
arXiv Detail & Related papers (2023-09-25T22:02:46Z) - AI, write an essay for me: A large-scale comparison of human-written
versus ChatGPT-generated essays [66.36541161082856]
ChatGPT and similar generative AI models have attracted hundreds of millions of users.
This study compares human-written versus ChatGPT-generated argumentative student essays.
arXiv Detail & Related papers (2023-04-24T12:58:28Z) - Artificial muses: Generative Artificial Intelligence Chatbots Have Risen
to Human-Level Creativity [1.332560004325655]
We compare human-generated ideas with those generated by six Generative Artificial Intelligence (GAI)
We found no qualitative difference between AI and human-generated creativity, although there are differences in how ideas are generated.
Our findings suggest that GAIs are valuable assistants in the creative process.
arXiv Detail & Related papers (2023-03-21T16:35:01Z) - Elaboration-Generating Commonsense Question Answering at Scale [77.96137534751445]
In question answering requiring common sense, language models (e.g., GPT-3) have been used to generate text expressing background knowledge.
We finetune smaller language models to generate useful intermediate context, referred to here as elaborations.
Our framework alternates between updating two language models -- an elaboration generator and an answer predictor -- allowing each to influence the other.
arXiv Detail & Related papers (2022-09-02T18:32:09Z) - CoAuthor: Designing a Human-AI Collaborative Writing Dataset for
Exploring Language Model Capabilities [92.79451009324268]
We present CoAuthor, a dataset designed for revealing GPT-3's capabilities in assisting creative and argumentative writing.
We demonstrate that CoAuthor can address questions about GPT-3's language, ideation, and collaboration capabilities.
We discuss how this work may facilitate a more principled discussion around LMs' promises and pitfalls in relation to interaction design.
arXiv Detail & Related papers (2022-01-18T07:51:57Z) - DeepCreativity: Measuring Creativity with Deep Learning Techniques [2.5426469613007012]
This paper explores the possibility of using generative learning techniques for automatic assessment of creativity.
We introduce a new measure, namely DeepCreativity, based on Margaret Boden's definition of creativity as composed by value, novelty and surprise.
arXiv Detail & Related papers (2022-01-16T19:00:01Z) - Telling Creative Stories Using Generative Visual Aids [52.623545341588304]
We asked writers to write creative stories from a starting prompt, and provided them with visuals created by generative AI models from the same prompt.
Compared to a control group, writers who used the visuals as story writing aid wrote significantly more creative, original, complete and visualizable stories.
Findings indicate that cross modality inputs by AI can benefit divergent aspects of creativity in human-AI co-creation, but hinders convergent thinking.
arXiv Detail & Related papers (2021-10-27T23:13:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.