Creativity Has Left the Chat: The Price of Debiasing Language Models

• URL: http://arxiv.org/abs/2406.05587v1
• Date: Sat, 8 Jun 2024 22:14:51 GMT
• Title: Creativity Has Left the Chat: The Price of Debiasing Language Models
• Authors: Behnam Mohammadi,
• Abstract summary: We investigate the unintended consequences of Reinforcement Learning from Human Feedback on the creativity of Large Language Models (LLMs) Our findings have significant implications for marketers who rely on LLMs for creative tasks such as copywriting, ad creation, and customer persona generation.
• Score: 1.223779595809275
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Models (LLMs) have revolutionized natural language processing but can exhibit biases and may generate toxic content. While alignment techniques like Reinforcement Learning from Human Feedback (RLHF) reduce these issues, their impact on creativity, defined as syntactic and semantic diversity, remains unexplored. We investigate the unintended consequences of RLHF on the creativity of LLMs through three experiments focusing on the Llama-2 series. Our findings reveal that aligned models exhibit lower entropy in token predictions, form distinct clusters in the embedding space, and gravitate towards "attractor states", indicating limited output diversity. Our findings have significant implications for marketers who rely on LLMs for creative tasks such as copywriting, ad creation, and customer persona generation. The trade-off between consistency and creativity in aligned models should be carefully considered when selecting the appropriate model for a given application. We also discuss the importance of prompt engineering in harnessing the creative potential of base models.

Related papers

• A linguistic analysis of undesirable outcomes in the era of generative AI [4.841442157674423]
  We present a comprehensive simulation framework built upon the chat version of LLama2, focusing on the linguistic aspects of the generated content. Our results show that the model produces less lexical rich content across generations, reducing diversity. We find that autophagy transforms the initial model into a more creative, doubtful and confused one, which might provide inaccurate answers.
  arXiv  Detail & Related papers  (2024-10-16T08:02:48Z)
• On the Modeling Capabilities of Large Language Models for Sequential Decision Making [52.128546842746246]
  Large pretrained models are showing increasingly better performance in reasoning and planning tasks. We evaluate their ability to produce decision-making policies, either directly, by generating actions, or indirectly. In environments with unfamiliar dynamics, we explore how fine-tuning LLMs with synthetic data can significantly improve their reward modeling capabilities.
  arXiv  Detail & Related papers  (2024-10-08T03:12:57Z)
• Benchmarking Language Model Creativity: A Case Study on Code Generation [17.56712029335294]
  creativity consists of at least two key characteristics: emphconvergent thinking (purposefulness to achieve a given goal) and emphdivergent thinking (adaptability to new environments or constraints) citeprunco 2003critical We introduce a framework for quantifying LLM creativity that incorporates the two characteristics. This is achieved by (1) Denial Prompting pushes LLMs to come up with more creative solutions to a given problem by incrementally imposing new constraints on the previous solution, and (2) defining and computing the NeoGauge metric which examines both convergent and divergent thinking in the generated creative
  arXiv  Detail & Related papers  (2024-07-12T05:55:22Z)
• Divergent Creativity in Humans and Large Language Models [37.67363469600804]
  The recent surge in the capabilities of Large Language Models has led to claims that they are approaching a level of creativity akin to human capabilities. We leverage recent advances in creativity science to build a framework for in-depth analysis of divergent creativity in both state-of-the-art LLMs and a substantial dataset of 100,000 humans.
  arXiv  Detail & Related papers  (2024-05-13T22:37:52Z)
• Probing the Creativity of Large Language Models: Can models produce divergent semantic association? [9.992602859777689]
  The present study aims to investigate the creative thinking of large language models through a cognitive perspective. We utilize the divergent association task ( DAT), an objective measurement of creativity that asks models to generate unrelated words and calculates the semantic distance between them. Our results imply that advanced large language models have divergent semantic associations, which is a fundamental process underlying creativity.
  arXiv  Detail & Related papers  (2023-10-17T11:23:32Z)
• A Mathematical Abstraction for Balancing the Trade-off Between Creativity and Reality in Large Language Models [35.25919932657958]
  Large Language Models are increasingly being used in domains such as generating prose, poetry or art. This work provides a mathematical abstraction to describe creativity and reality based on certain losses. A model trained on these losses balances the trade-off between the creativity and reality of the model.
  arXiv  Detail & Related papers  (2023-06-04T08:12:34Z)
• UniDiff: Advancing Vision-Language Models with Generative and Discriminative Learning [86.91893533388628]
  This paper presents UniDiff, a unified multi-modal model that integrates image-text contrastive learning (ITC), text-conditioned image synthesis learning (IS), and reciprocal semantic consistency modeling (RSC) UniDiff demonstrates versatility in both multi-modal understanding and generative tasks.
  arXiv  Detail & Related papers  (2023-06-01T15:39:38Z)
• CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing [139.77117915309023]
  CRITIC allows large language models to validate and amend their own outputs in a manner similar to human interaction with tools. Comprehensive evaluations involving free-form question answering, mathematical program synthesis, and toxicity reduction demonstrate that CRITIC consistently enhances the performance of LLMs.
  arXiv  Detail & Related papers  (2023-05-19T15:19:44Z)
• Large Language Models with Controllable Working Memory [64.71038763708161]
  Large language models (LLMs) have led to a series of breakthroughs in natural language processing (NLP) What further sets these models apart is the massive amounts of world knowledge they internalize during pretraining. How the model's world knowledge interacts with the factual information presented in the context remains under explored.
  arXiv  Detail & Related papers  (2022-11-09T18:58:29Z)
• Towards Creativity Characterization of Generative Models via Group-based Subset Scanning [64.6217849133164]
  We propose group-based subset scanning to identify, quantify, and characterize creative processes. We find that creative samples generate larger subsets of anomalies than normal or non-creative samples across datasets.
  arXiv  Detail & Related papers  (2022-03-01T15:07:14Z)
• Towards creativity characterization of generative models via group-based subset scanning [51.84144826134919]
  We propose group-based subset scanning to quantify, detect, and characterize creative processes. Creative samples generate larger subsets of anomalies than normal or non-creative samples across datasets.
  arXiv  Detail & Related papers  (2021-04-01T14:07:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.