ChatGPT Role-play Dataset: Analysis of User Motives and Model Naturalness

• URL: http://arxiv.org/abs/2403.18121v1
• Date: Tue, 26 Mar 2024 22:01:13 GMT
• Title: ChatGPT Role-play Dataset: Analysis of User Motives and Model Naturalness
• Authors: Yufei Tao, Ameeta Agrawal, Judit Dombi, Tetyana Sydorenko, Jung In Lee,
• Abstract summary: We study how ChatGPT behaves during conversations in different settings by analyzing its interactions in both a normal way and a role-play setting. Our study highlights the diversity of user motives when interacting with ChatGPT and variable AI naturalness, showing not only the nuanced dynamics of natural conversations between humans and AI, but also providing new avenues for improving the effectiveness of human-AI communication.
• Score: 4.564433526993029
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Recent advances in interactive large language models like ChatGPT have revolutionized various domains; however, their behavior in natural and role-play conversation settings remains underexplored. In our study, we address this gap by deeply investigating how ChatGPT behaves during conversations in different settings by analyzing its interactions in both a normal way and a role-play setting. We introduce a novel dataset of broad range of human-AI conversations annotated with user motives and model naturalness to examine (i) how humans engage with the conversational AI model, and (ii) how natural are AI model responses. Our study highlights the diversity of user motives when interacting with ChatGPT and variable AI naturalness, showing not only the nuanced dynamics of natural conversations between humans and AI, but also providing new avenues for improving the effectiveness of human-AI communication.

Related papers

• From Pixels to Personas: Investigating and Modeling Self-Anthropomorphism in Human-Robot Dialogues [29.549159419088294]
  Self-anthropomorphism in robots manifests itself through their display of human-like characteristics in dialogue, such as expressing preferences and emotions. We show significant differences in these two types of responses and propose transitioning from one type to the other. We introduce Pix2Persona, a novel dataset aimed at developing ethical and engaging AI systems in various embodiments.
  arXiv  Detail & Related papers  (2024-10-04T19:06:24Z)
• Self-Directed Turing Test for Large Language Models [56.64615470513102]
  The Turing test examines whether AIs can exhibit human-like behaviour in natural language conversations. Traditional Turing tests adopt a rigid dialogue format where each participant sends only one message each time. This paper proposes the Self-Directed Turing Test, which extends the original test with a burst dialogue format.
  arXiv  Detail & Related papers  (2024-08-19T09:57:28Z)
• Designing and Evaluating Multi-Chatbot Interface for Human-AI Communication: Preliminary Findings from a Persuasion Task [1.360607903399872]
  This study examines the impact of multi-chatbot communication in a specific persuasion setting: promoting charitable donations. We developed an online environment that enables multi-chatbot communication and conducted a pilot experiment. We present our development process of the multi-chatbot interface and present preliminary findings from a pilot experiment.
  arXiv  Detail & Related papers  (2024-06-28T04:33:41Z)
• A Linguistic Comparison between Human and ChatGPT-Generated Conversations [9.022590646680095]
  The research employs Linguistic Inquiry and Word Count analysis, comparing ChatGPT-generated conversations with human conversations. Results show greater variability and authenticity in human dialogues, but ChatGPT excels in categories such as social processes, analytical style, cognition, attentional focus, and positive emotional tone.
  arXiv  Detail & Related papers  (2024-01-29T21:43:27Z)
• Synocene, Beyond the Anthropocene: De-Anthropocentralising Human-Nature-AI Interaction [0.27488316163114823]
  This case study presents a pioneering exploration into the AI attitudes (ecocentric, anthropocentric and antipathetic) toward nature. We conducted a real-life experiment in which participants underwent an immersive de-anthropocentric experience in a forest. By creating fictional AI characters with ecocentric attributes, emotions and views, we successfully amplified ecocentric exchanges.
  arXiv  Detail & Related papers  (2023-12-13T11:04:06Z)
• Tachikuma: Understading Complex Interactions with Multi-Character and Novel Objects by Large Language Models [67.20964015591262]
  We introduce a benchmark named Tachikuma, comprising a Multiple character and novel Object based interaction Estimation task and a supporting dataset. The dataset captures log data from real-time communications during gameplay, providing diverse, grounded, and complex interactions for further explorations. We present a simple prompting baseline and evaluate its performance, demonstrating its effectiveness in enhancing interaction understanding.
  arXiv  Detail & Related papers  (2023-07-24T07:40:59Z)
• Interactive Natural Language Processing [67.87925315773924]
  Interactive Natural Language Processing (iNLP) has emerged as a novel paradigm within the field of NLP. This paper offers a comprehensive survey of iNLP, starting by proposing a unified definition and framework of the concept.
  arXiv  Detail & Related papers  (2023-05-22T17:18:29Z)
• PLACES: Prompting Language Models for Social Conversation Synthesis [103.94325597273316]
  We use a small set of expert-written conversations as in-context examples to synthesize a social conversation dataset using prompting. We perform several thorough evaluations of our synthetic conversations compared to human-collected conversations.
  arXiv  Detail & Related papers  (2023-02-07T05:48:16Z)
• A Probabilistic Model Of Interaction Dynamics for Dyadic Face-to-Face Settings [1.9544213396776275]
  We develop a probabilistic model to capture the interaction dynamics between pairs of participants in a face-to-face setting. This interaction encoding is then used to influence the generation when predicting one agent's future dynamics. We show that our model successfully delineates between the modes, based on their interacting dynamics.
  arXiv  Detail & Related papers  (2022-07-10T23:31:27Z)
• DialogueNeRF: Towards Realistic Avatar Face-to-Face Conversation Video Generation [54.84137342837465]
  Face-to-face conversations account for the vast majority of daily conversations. Most existing methods focused on single-person talking head generation. We propose a novel unified framework based on neural radiance field (NeRF)
  arXiv  Detail & Related papers  (2022-03-15T14:16:49Z)
• You Impress Me: Dialogue Generation via Mutual Persona Perception [62.89449096369027]
  The research in cognitive science suggests that understanding is an essential signal for a high-quality chit-chat conversation. Motivated by this, we propose P2 Bot, a transmitter-receiver based framework with the aim of explicitly modeling understanding.
  arXiv  Detail & Related papers  (2020-04-11T12:51:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.