Creativity Support in the Age of Large Language Models: An Empirical
Study Involving Emerging Writers
- URL: http://arxiv.org/abs/2309.12570v3
- Date: Tue, 30 Jan 2024 15:56:47 GMT
- Title: Creativity Support in the Age of Large Language Models: An Empirical
Study Involving Emerging Writers
- Authors: Tuhin Chakrabarty, Vishakh Padmakumar, Faeze Brahman, Smaranda Muresan
- Abstract summary: We investigate the utility of modern large language models in assisting professional writers via an empirical user study.
We find that while writers seek LLM's help across all three types of cognitive activities, they find LLMs more helpful in translation and reviewing.
- Score: 33.3564201174124
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The development of large language models (LLMs) capable of following
instructions and engaging in conversational interactions sparked increased
interest in their utilization across various support tools. We investigate the
utility of modern LLMs in assisting professional writers via an empirical user
study (n=30). The design of our collaborative writing interface is grounded in
the cognitive process model of writing that views writing as a goal-oriented
thinking process encompassing non-linear cognitive activities: planning,
translating, and reviewing. Participants are asked to submit a post-completion
survey to provide feedback on the potential and pitfalls of LLMs as writing
collaborators. Upon analyzing the writer-LLM interactions, we find that while
writers seek LLM's help across all three types of cognitive activities, they
find LLMs more helpful in translation and reviewing. Our findings from
analyzing both the interactions and the survey responses highlight future
research directions in creative writing assistance using LLMs.
Related papers
- I Need Help! Evaluating LLM's Ability to Ask for Users' Support: A Case Study on Text-to-SQL Generation [60.00337758147594]
We propose metrics to evaluate the trade-off between performance improvements and user burden.
Our experiments reveal that without external feedback, many LLMs struggle to recognize their need for additional support.
arXiv Detail & Related papers (2024-07-20T06:12:29Z) - Inclusivity in Large Language Models: Personality Traits and Gender Bias in Scientific Abstracts [49.97673761305336]
We evaluate three large language models (LLMs) for their alignment with human narrative styles and potential gender biases.
Our findings indicate that, while these models generally produce text closely resembling human authored content, variations in stylistic features suggest significant gender biases.
arXiv Detail & Related papers (2024-06-27T19:26:11Z) - LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing [106.45895712717612]
Large language models (LLMs) have shown remarkable versatility in various generative tasks.
This study focuses on the topic of LLMs assist NLP Researchers.
To our knowledge, this is the first work to provide such a comprehensive analysis.
arXiv Detail & Related papers (2024-06-24T01:30:22Z) - Apprentices to Research Assistants: Advancing Research with Large Language Models [0.0]
Large Language Models (LLMs) have emerged as powerful tools in various research domains.
This article examines their potential through a literature review and firsthand experimentation.
arXiv Detail & Related papers (2024-04-09T15:53:06Z) - KIWI: A Dataset of Knowledge-Intensive Writing Instructions for
Answering Research Questions [63.307317584926146]
Large language models (LLMs) adapted to follow user instructions are now widely deployed as conversational agents.
In this work, we examine one increasingly common instruction-following task: providing writing assistance to compose a long-form answer.
We construct KIWI, a dataset of knowledge-intensive writing instructions in the scientific domain.
arXiv Detail & Related papers (2024-03-06T17:16:44Z) - Next-Step Hint Generation for Introductory Programming Using Large
Language Models [0.8002196839441036]
Large Language Models possess skills such as answering questions, writing essays or solving programming exercises.
This work explores how LLMs can contribute to programming education by supporting students with automated next-step hints.
arXiv Detail & Related papers (2023-12-03T17:51:07Z) - Evaluating Large Language Model Creativity from a Literary Perspective [13.672268920902187]
This paper assesses the potential for large language models to serve as assistive tools in the creative writing process.
We develop interactive and multi-voice prompting strategies that interleave background descriptions, instructions that guide composition, samples of text in the target style, and critical discussion of the given samples.
arXiv Detail & Related papers (2023-11-30T16:46:25Z) - "It Felt Like Having a Second Mind": Investigating Human-AI
Co-creativity in Prewriting with Large Language Models [20.509651636971864]
This study investigates human-LLM collaboration patterns and dynamics during prewriting.
During collaborative prewriting, there appears to be a three-stage iterative Human-AI Co-creativity process.
arXiv Detail & Related papers (2023-07-20T16:55:25Z) - Improving Factuality and Reasoning in Language Models through Multiagent
Debate [95.10641301155232]
We present a complementary approach to improve language responses where multiple language model instances propose and debate their individual responses and reasoning processes over multiple rounds to arrive at a common final answer.
Our findings indicate that this approach significantly enhances mathematical and strategic reasoning across a number of tasks.
Our approach may be directly applied to existing black-box models and uses identical procedure and prompts for all tasks we investigate.
arXiv Detail & Related papers (2023-05-23T17:55:11Z) - Approach Intelligent Writing Assistants Usability with Seven Stages of
Action [9.378355457555319]
We adopt Norman's seven stages of action as a framework to approach the interaction design of intelligent writing assistants.
We illustrate the framework's applicability to writing tasks by providing an example of software tutorial authoring.
arXiv Detail & Related papers (2023-04-06T02:11:55Z) - CoAuthor: Designing a Human-AI Collaborative Writing Dataset for
Exploring Language Model Capabilities [92.79451009324268]
We present CoAuthor, a dataset designed for revealing GPT-3's capabilities in assisting creative and argumentative writing.
We demonstrate that CoAuthor can address questions about GPT-3's language, ideation, and collaboration capabilities.
We discuss how this work may facilitate a more principled discussion around LMs' promises and pitfalls in relation to interaction design.
arXiv Detail & Related papers (2022-01-18T07:51:57Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.