Improving Proactive Dialog Agents Using Socially-Aware Reinforcement Learning

• URL: http://arxiv.org/abs/2211.15359v2
• Date: Thu, 22 Jun 2023 08:55:12 GMT
• Title: Improving Proactive Dialog Agents Using Socially-Aware Reinforcement Learning
• Authors: Matthias Kraus, Nicolas Wagner, Ron Riekenbrauck and Wolfgang Minker
• Abstract summary: Well-defined proactive behavior may improve human-machine cooperation. We propose a novel approach including both social as well as task-relevant features in the dialog.
• Score: 3.9011896000134825
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: The next step for intelligent dialog agents is to escape their role as silent bystanders and become proactive. Well-defined proactive behavior may improve human-machine cooperation, as the agent takes a more active role during interaction and takes off responsibility from the user. However, proactivity is a double-edged sword because poorly executed pre-emptive actions may have a devastating effect not only on the task outcome but also on the relationship with the user. For designing adequate proactive dialog strategies, we propose a novel approach including both social as well as task-relevant features in the dialog. Here, the primary goal is to optimize proactive behavior so that it is task-oriented - this implies high task success and efficiency - while also being socially effective by fostering user trust. Including both aspects in the reward function for training a proactive dialog agent using reinforcement learning showed the benefit of our approach for more successful human-machine cooperation.

Related papers

• ReSpAct: Harmonizing Reasoning, Speaking, and Acting Towards Building Large Language Model-Based Conversational AI Agents [11.118991548784459]
  Large language model (LLM)-based agents have been increasingly used to interact with external environments. Current frameworks do not enable these agents to work with users and interact with them to align on the details of their tasks. This work introduces ReSpAct, a novel framework that combines the essential skills for building task-oriented "conversational" agents.
  arXiv  Detail & Related papers  (2024-11-01T15:57:45Z)
• CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society [58.04479313658851]
  This paper explores the potential of building scalable techniques to facilitate autonomous cooperation among communicative agents. We propose a novel communicative agent framework named role-playing. Our contributions include introducing a novel communicative agent framework, offering a scalable approach for studying the cooperative behaviors and capabilities of multi-agent systems.
  arXiv  Detail & Related papers  (2023-03-31T01:09:00Z)
• Does It Affect You? Social and Learning Implications of Using Cognitive-Affective State Recognition for Proactive Human-Robot Tutoring [4.384546153204966]
  This paper investigates how the student's cognitive-affective states can be used by a robotic tutor for triggering proactive tutoring dialogue. We studied whether the initiation of proactive behaviour after the detection of signs of confusion improves the student's concentration and trust in the agent. The results show that high proactive behaviour harms trust, especially when triggered during negative cognitive-affective states.
  arXiv  Detail & Related papers  (2022-12-20T15:31:58Z)
• Aligning to Social Norms and Values in Interactive Narratives [89.82264844526333]
  We focus on creating agents that act in alignment with socially beneficial norms and values in interactive narratives or text-based games. We introduce the GALAD agent that uses the social commonsense knowledge present in specially trained language models to contextually restrict its action space to only those actions that are aligned with socially beneficial values.
  arXiv  Detail & Related papers  (2022-05-04T09:54:33Z)
• Interacting with Non-Cooperative User: A New Paradigm for Proactive Dialogue Policy [83.61404191470126]
  We propose a new solution named I-Pro that can learn Proactive policy in the Interactive setting. Specifically, we learn the trade-off via a learned goal weight, which consists of four factors. The experimental results demonstrate I-Pro significantly outperforms baselines in terms of effectiveness and interpretability.
  arXiv  Detail & Related papers  (2022-04-07T14:11:31Z)
• User Satisfaction Estimation with Sequential Dialogue Act Modeling in Goal-oriented Conversational Systems [65.88679683468143]
  We propose a novel framework, namely USDA, to incorporate the sequential dynamics of dialogue acts for predicting user satisfaction. USDA incorporates the sequential transitions of both content and act features in the dialogue to predict the user satisfaction. Experimental results on four benchmark goal-oriented dialogue datasets show that the proposed method substantially and consistently outperforms existing methods on USE.
  arXiv  Detail & Related papers  (2022-02-07T02:50:07Z)
• Interactive Inverse Reinforcement Learning for Cooperative Games [7.257751371276486]
  We study the problem of designing AI agents that can learn to cooperate effectively with a potentially suboptimal partner. This problem is modeled as a cooperative episodic two-agent Markov decision process. We show that when the learning agent's policies have a significant effect on the transition function, the reward function can be learned efficiently.
  arXiv  Detail & Related papers  (2021-11-08T18:24:52Z)
• What Does The User Want? Information Gain for Hierarchical Dialogue Policy Optimisation [3.1433893853959605]
  optimisation via reinforcement learning (RL) is susceptible to sample inefficiency and instability. We propose the usage of an intrinsic reward based on information gain to address this issue. Our algorithm, which we call FeudalGain, achieves state-of-the-art results in most environments of the PyDial framework.
  arXiv  Detail & Related papers  (2021-09-15T07:21:26Z)
• Can You be More Social? Injecting Politeness and Positivity into Task-Oriented Conversational Agents [60.27066549589362]
  Social language used by human agents is associated with greater users' responsiveness and task completion. The model uses a sequence-to-sequence deep learning architecture, extended with a social language understanding element. Evaluation in terms of content preservation and social language level using both human judgment and automatic linguistic measures shows that the model can generate responses that enable agents to address users' issues in a more socially appropriate way.
  arXiv  Detail & Related papers  (2020-12-29T08:22:48Z)
• Rethinking Supervised Learning and Reinforcement Learning in Task-Oriented Dialogue Systems [58.724629408229205]
  We demonstrate how traditional supervised learning and a simulator-free adversarial learning method can be used to achieve performance comparable to state-of-the-art RL-based methods. Our main goal is not to beat reinforcement learning with supervised learning, but to demonstrate the value of rethinking the role of reinforcement learning and supervised learning in optimizing task-oriented dialogue systems.
  arXiv  Detail & Related papers  (2020-09-21T12:04:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.