Creativity Support in the Age of Large Language Models: An Empirical Study Involving Emerging Writers

• URL: http://arxiv.org/abs/2309.12570v3
• Date: Tue, 30 Jan 2024 15:56:47 GMT
• Title: Creativity Support in the Age of Large Language Models: An Empirical Study Involving Emerging Writers
• Authors: Tuhin Chakrabarty, Vishakh Padmakumar, Faeze Brahman, Smaranda Muresan
• Abstract summary: We investigate the utility of modern large language models in assisting professional writers via an empirical user study. We find that while writers seek LLM's help across all three types of cognitive activities, they find LLMs more helpful in translation and reviewing.
• Score: 33.3564201174124
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The development of large language models (LLMs) capable of following instructions and engaging in conversational interactions sparked increased interest in their utilization across various support tools. We investigate the utility of modern LLMs in assisting professional writers via an empirical user study (n=30). The design of our collaborative writing interface is grounded in the cognitive process model of writing that views writing as a goal-oriented thinking process encompassing non-linear cognitive activities: planning, translating, and reviewing. Participants are asked to submit a post-completion survey to provide feedback on the potential and pitfalls of LLMs as writing collaborators. Upon analyzing the writer-LLM interactions, we find that while writers seek LLM's help across all three types of cognitive activities, they find LLMs more helpful in translation and reviewing. Our findings from analyzing both the interactions and the survey responses highlight future research directions in creative writing assistance using LLMs.

Related papers

• Understanding the Human-LLM Dynamic: A Literature Survey of LLM Use in Programming Tasks [0.850206009406913]
  Large Language Models (LLMs) are transforming programming practices, offering significant capabilities for code generation activities. This paper focuses on their use in programming tasks, drawing insights from user studies that assess the impact of LLMs on programming tasks.
  arXiv  Detail & Related papers  (2024-10-01T19:34:46Z)
• What is the Role of Large Language Models in the Evolution of Astronomy Research? [0.0]
  ChatGPT and other state-of-the-art large language models (LLMs) are rapidly transforming multiple fields. These models, commonly trained on vast datasets, exhibit human-like text generation capabilities.
  arXiv  Detail & Related papers  (2024-09-30T12:42:25Z)
• Exploring Knowledge Tracing in Tutor-Student Dialogues [53.52699766206808]
  We present a first attempt at performing knowledge tracing (KT) in tutor-student dialogues. We propose methods to identify the knowledge components/skills involved in each dialogue turn. We then apply a range of KT methods on the resulting labeled data to track student knowledge levels over an entire dialogue.
  arXiv  Detail & Related papers  (2024-09-24T22:31:39Z)
• What You Need is What You Get: Theory of Mind for an LLM-Based Code Understanding Assistant [0.0]
  A growing number of tools have used Large Language Models (LLMs) to support developers' code understanding. In this study, we designed an LLM-based conversational assistant that provides a personalized interaction based on inferred user mental state. Our results provide insights for researchers and tool builders who want to create or improve LLM-based conversational assistants to support novices in code understanding.
  arXiv  Detail & Related papers  (2024-08-08T14:08:15Z)
• Inclusivity in Large Language Models: Personality Traits and Gender Bias in Scientific Abstracts [49.97673761305336]
  We evaluate three large language models (LLMs) for their alignment with human narrative styles and potential gender biases. Our findings indicate that, while these models generally produce text closely resembling human authored content, variations in stylistic features suggest significant gender biases.
  arXiv  Detail & Related papers  (2024-06-27T19:26:11Z)
• LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing [106.45895712717612]
  Large language models (LLMs) have shown remarkable versatility in various generative tasks. This study focuses on the topic of LLMs assist NLP Researchers. To our knowledge, this is the first work to provide such a comprehensive analysis.
  arXiv  Detail & Related papers  (2024-06-24T01:30:22Z)
• Apprentices to Research Assistants: Advancing Research with Large Language Models [0.0]
  Large Language Models (LLMs) have emerged as powerful tools in various research domains. This article examines their potential through a literature review and firsthand experimentation.
  arXiv  Detail & Related papers  (2024-04-09T15:53:06Z)
• KIWI: A Dataset of Knowledge-Intensive Writing Instructions for Answering Research Questions [63.307317584926146]
  Large language models (LLMs) adapted to follow user instructions are now widely deployed as conversational agents. In this work, we examine one increasingly common instruction-following task: providing writing assistance to compose a long-form answer. We construct KIWI, a dataset of knowledge-intensive writing instructions in the scientific domain.
  arXiv  Detail & Related papers  (2024-03-06T17:16:44Z)
• "It Felt Like Having a Second Mind": Investigating Human-AI Co-creativity in Prewriting with Large Language Models [20.509651636971864]
  This study investigates human-LLM collaboration patterns and dynamics during prewriting. During collaborative prewriting, there appears to be a three-stage iterative Human-AI Co-creativity process.
  arXiv  Detail & Related papers  (2023-07-20T16:55:25Z)
• Improving Factuality and Reasoning in Language Models through Multiagent Debate [95.10641301155232]
  We present a complementary approach to improve language responses where multiple language model instances propose and debate their individual responses and reasoning processes over multiple rounds to arrive at a common final answer. Our findings indicate that this approach significantly enhances mathematical and strategic reasoning across a number of tasks. Our approach may be directly applied to existing black-box models and uses identical procedure and prompts for all tasks we investigate.
  arXiv  Detail & Related papers  (2023-05-23T17:55:11Z)
• CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities [92.79451009324268]
  We present CoAuthor, a dataset designed for revealing GPT-3's capabilities in assisting creative and argumentative writing. We demonstrate that CoAuthor can address questions about GPT-3's language, ideation, and collaboration capabilities. We discuss how this work may facilitate a more principled discussion around LMs' promises and pitfalls in relation to interaction design.
  arXiv  Detail & Related papers  (2022-01-18T07:51:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.