AI Alignment Dialogues: An Interactive Approach to AI Alignment in Support Agents

• URL: http://arxiv.org/abs/2301.06421v2
• Date: Thu, 5 Oct 2023 11:15:23 GMT
• Title: AI Alignment Dialogues: An Interactive Approach to AI Alignment in Support Agents
• Authors: Pei-Yu Chen, Myrthe L. Tielman, Dirk K.J. Heylen, Catholijn M. Jonker, M. Birna van Riemsdijk
• Abstract summary: This paper proposes a new approach to AI alignment: alignment dialogues with which users and agents try to achieve and maintain alignment via interaction. We argue that alignment dialogues have a number of advantages in comparison to data-driven approaches. The advantages of alignment dialogues include allowing the users to directly convey higher-level concepts to the agent, and making the agent more transparent and trustworthy.
• Score: 3.0731004832223796
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: AI alignment is about ensuring AI systems only pursue goals and activities that are beneficial to humans. Most of the current approach to AI alignment is to learn what humans value from their behavioural data. This paper proposes a different way of looking at the notion of alignment, namely by introducing AI Alignment Dialogues: dialogues with which users and agents try to achieve and maintain alignment via interaction. We argue that alignment dialogues have a number of advantages in comparison to data-driven approaches, especially for behaviour support agents, which aim to support users in achieving their desired future behaviours rather than their current behaviours. The advantages of alignment dialogues include allowing the users to directly convey higher-level concepts to the agent, and making the agent more transparent and trustworthy. In this paper we outline the concept and high-level structure of alignment dialogues. Moreover, we conducted a qualitative focus group user study from which we developed a model that describes how alignment dialogues affect users, and created design suggestions for AI alignment dialogues. Through this we establish foundations for AI alignment dialogues and shed light on what requires further development and research.

Related papers

• Interactive Dialogue Agents via Reinforcement Learning on Hindsight Regenerations [58.65755268815283]
  Many real dialogues are interactive, meaning an agent's utterances will influence their conversational partner, elicit information, or change their opinion. We use this fact to rewrite and augment existing suboptimal data, and train via offline reinforcement learning (RL) an agent that outperforms both prompting and learning from unaltered human demonstrations. Our results in a user study with real humans show that our approach greatly outperforms existing state-of-the-art dialogue agents.
  arXiv  Detail & Related papers  (2024-11-07T21:37:51Z)
• Towards Bidirectional Human-AI Alignment: A Systematic Review for Clarifications, Framework, and Future Directions [101.67121669727354]
  Recent advancements in AI have highlighted the importance of guiding AI systems towards the intended goals, ethical principles, and values of individuals and groups, a concept broadly recognized as alignment. The lack of clarified definitions and scopes of human-AI alignment poses a significant obstacle, hampering collaborative efforts across research domains to achieve this alignment. We introduce a systematic review of over 400 papers published between 2019 and January 2024, spanning multiple domains such as Human-Computer Interaction (HCI), Natural Language Processing (NLP), Machine Learning (ML)
  arXiv  Detail & Related papers  (2024-06-13T16:03:25Z)
• Interactive AI Alignment: Specification, Process, and Evaluation Alignment [30.599781014726823]
  Modern AI enables a high-level, declarative form of interaction. Users describe the intended outcome they wish an AI to produce, but do not actually create the outcome themselves.
  arXiv  Detail & Related papers  (2023-10-23T14:33:11Z)
• Dialogue Agents 101: A Beginner's Guide to Critical Ingredients for Designing Effective Conversational Systems [29.394466123216258]
  This study provides a comprehensive overview of the primary characteristics of a dialogue agent, their corresponding open-domain datasets, and the methods used to benchmark these datasets. We propose UNIT, a UNified dIalogue dataseT constructed from conversations of existing datasets for different dialogue tasks capturing the nuances for each of them.
  arXiv  Detail & Related papers  (2023-07-14T10:05:47Z)
• Target-Guided Dialogue Response Generation Using Commonsense and Data Augmentation [32.764356638437214]
  We introduce a new technique for target-guided response generation. We also propose techniques to re-purpose existing dialogue datasets for target-guided generation. Our work generally enables dialogue system designers to exercise more control over the conversations that their systems produce.
  arXiv  Detail & Related papers  (2022-05-19T04:01:40Z)
• Interacting with Non-Cooperative User: A New Paradigm for Proactive Dialogue Policy [83.61404191470126]
  We propose a new solution named I-Pro that can learn Proactive policy in the Interactive setting. Specifically, we learn the trade-off via a learned goal weight, which consists of four factors. The experimental results demonstrate I-Pro significantly outperforms baselines in terms of effectiveness and interpretability.
  arXiv  Detail & Related papers  (2022-04-07T14:11:31Z)
• Is this Dialogue Coherent? Learning from Dialogue Acts and Entities [82.44143808977209]
  We create the Switchboard Coherence (SWBD-Coh) corpus, a dataset of human-human spoken dialogues annotated with turn coherence ratings. Our statistical analysis of the corpus indicates how turn coherence perception is affected by patterns of distribution of entities. We find that models combining both DA and entity information yield the best performances both for response selection and turn coherence rating.
  arXiv  Detail & Related papers  (2020-06-17T21:02:40Z)
• Contextual Dialogue Act Classification for Open-Domain Conversational Agents [10.576497782941697]
  Classifying the general intent of the user utterance in a conversation, also known as Dialogue Act (DA), is a key step in Natural Language Understanding (NLU) for conversational agents. We propose CDAC (Contextual Dialogue Act), a simple yet effective deep learning approach for contextual dialogue act classification. We use transfer learning to adapt models trained on human-human conversations to predict dialogue acts in human-machine dialogues.
  arXiv  Detail & Related papers  (2020-05-28T06:48:10Z)
• Will I Sound Like Me? Improving Persona Consistency in Dialogues through Pragmatic Self-Consciousness [62.55060760615656]
  Recent models tackling consistency often train with additional Natural Language Inference (NLI) labels or attach trained extra modules to the generative agent for maintaining consistency. Inspired by social cognition and pragmatics, we endow existing dialogue agents with public self-consciousness on the fly through an imaginary listener. Our approach, based on the Rational Speech Acts framework, can enforce dialogue agents to refrain from uttering contradiction.
  arXiv  Detail & Related papers  (2020-04-13T08:16:16Z)
• You Impress Me: Dialogue Generation via Mutual Persona Perception [62.89449096369027]
  The research in cognitive science suggests that understanding is an essential signal for a high-quality chit-chat conversation. Motivated by this, we propose P2 Bot, a transmitter-receiver based framework with the aim of explicitly modeling understanding.
  arXiv  Detail & Related papers  (2020-04-11T12:51:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.