LLM Agents in Interaction: Measuring Personality Consistency and Linguistic Alignment in Interacting Populations of Large Language Models

• URL: http://arxiv.org/abs/2402.02896v1
• Date: Mon, 5 Feb 2024 11:05:20 GMT
• Title: LLM Agents in Interaction: Measuring Personality Consistency and Linguistic Alignment in Interacting Populations of Large Language Models
• Authors: Ivar Frisch, Mario Giulianelli
• Abstract summary: We create a two-group population of large language models (LLMs) agents using a simple variability-inducing sampling algorithm. We administer personality tests and submit the agents to a collaborative writing task, finding that different profiles exhibit different degrees of personality consistency and linguistic alignment to their conversational partners.
• Score: 4.706971067968811
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While both agent interaction and personalisation are vibrant topics in research on large language models (LLMs), there has been limited focus on the effect of language interaction on the behaviour of persona-conditioned LLM agents. Such an endeavour is important to ensure that agents remain consistent to their assigned traits yet are able to engage in open, naturalistic dialogues. In our experiments, we condition GPT-3.5 on personality profiles through prompting and create a two-group population of LLM agents using a simple variability-inducing sampling algorithm. We then administer personality tests and submit the agents to a collaborative writing task, finding that different profiles exhibit different degrees of personality consistency and linguistic alignment to their conversational partners. Our study seeks to lay the groundwork for better understanding of dialogue-based interaction between LLMs and highlights the need for new approaches to crafting robust, more human-like LLM personas for interactive environments.

Related papers

• NewsInterview: a Dataset and a Playground to Evaluate LLMs' Ground Gap via Informational Interviews [65.35458530702442]
  We focus on journalistic interviews, a domain rich in grounding communication and abundant in data. We curate a dataset of 40,000 two-person informational interviews from NPR and CNN. LLMs are significantly less likely than human interviewers to use acknowledgements and to pivot to higher-level questions.
  arXiv  Detail & Related papers  (2024-11-21T01:37:38Z)
• Interactive Dialogue Agents via Reinforcement Learning on Hindsight Regenerations [58.65755268815283]
  Many real dialogues are interactive, meaning an agent's utterances will influence their conversational partner, elicit information, or change their opinion. We use this fact to rewrite and augment existing suboptimal data, and train via offline reinforcement learning (RL) an agent that outperforms both prompting and learning from unaltered human demonstrations. Our results in a user study with real humans show that our approach greatly outperforms existing state-of-the-art dialogue agents.
  arXiv  Detail & Related papers  (2024-11-07T21:37:51Z)
• Spontaneous Emergence of Agent Individuality through Social Interactions in LLM-Based Communities [0.0]
  We study the emergence of agency from scratch by using Large Language Model (LLM)-based agents. By analyzing this multi-agent simulation, we report valuable new insights into how social norms, cooperation, and personality traits can emerge spontaneously.
  arXiv  Detail & Related papers  (2024-11-05T16:49:33Z)
• LMLPA: Language Model Linguistic Personality Assessment [11.599282127259736]
  Large Language Models (LLMs) are increasingly used in everyday life and research. measuring the personality of a given LLM is currently a challenge. This paper introduces the Language Model Linguistic Personality Assessment (LMLPA), a system designed to evaluate the linguistic personalities of LLMs.
  arXiv  Detail & Related papers  (2024-10-23T07:48:51Z)
• Fostering Natural Conversation in Large Language Models with NICO: a Natural Interactive COnversation dataset [28.076028584051617]
  We introduce NICO, a Natural Interactive COnversation dataset in Chinese. We first use GPT-4-turbo to generate dialogue drafts and make them cover 20 daily-life topics and 5 types of social interactions. We define two dialogue-level natural conversation tasks and two sentence-level tasks for identifying and rewriting unnatural sentences.
  arXiv  Detail & Related papers  (2024-08-18T02:06:25Z)
• PersLLM: A Personified Training Approach for Large Language Models [66.16513246245401]
  We propose PersLLM, integrating psychology-grounded principles of personality: social practice, consistency, and dynamic development. We incorporate personality traits directly into the model parameters, enhancing the model's resistance to induction, promoting consistency, and supporting the dynamic evolution of personality.
  arXiv  Detail & Related papers  (2024-07-17T08:13:22Z)
• Rel-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliance [73.19687314438133]
  We study how reliance is affected by contextual features of an interaction. We find that contextual characteristics significantly affect human reliance behavior. Our results show that calibration and language quality alone are insufficient in evaluating the risks of human-LM interactions.
  arXiv  Detail & Related papers  (2024-07-10T18:00:05Z)
• CloChat: Understanding How People Customize, Interact, and Experience Personas in Large Language Models [15.915071948354466]
  CloChat is an interface supporting easy and accurate customization of agent personas in large language models. Results indicate that participants formed emotional bonds with the customized agents, engaged in more dynamic dialogues, and showed interest in sustaining interactions.
  arXiv  Detail & Related papers  (2024-02-23T11:25:17Z)
• Zero-Shot Goal-Directed Dialogue via RL on Imagined Conversations [70.7884839812069]
  Large language models (LLMs) have emerged as powerful and general solutions to many natural language tasks. However, many of the most important applications of language generation are interactive, where an agent has to talk to a person to reach a desired outcome. In this work, we explore a new method for adapting LLMs with RL for such goal-directed dialogue.
  arXiv  Detail & Related papers  (2023-11-09T18:45:16Z)
• Interactive Natural Language Processing [67.87925315773924]
  Interactive Natural Language Processing (iNLP) has emerged as a novel paradigm within the field of NLP. This paper offers a comprehensive survey of iNLP, starting by proposing a unified definition and framework of the concept.
  arXiv  Detail & Related papers  (2023-05-22T17:18:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.