Tailored Conversations beyond LLMs: A RL-Based Dialogue Manager

• URL: http://arxiv.org/abs/2506.19652v1
• Date: Tue, 24 Jun 2025 14:15:26 GMT
• Title: Tailored Conversations beyond LLMs: A RL-Based Dialogue Manager
• Authors: Lucie Galland, Catherine Pelachaud, Florian Pecune,
• Abstract summary: We propose a framework that integrates large language models (LLMs) with an RL-based dialogue manager for open-ended dialogue with a specific goal.<n>We employ hierarchical reinforcement learning to model the structured phases of dialogue and employ meta-learning to enhance adaptability across diverse user profiles.
• Score: 0.7499722271664147
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In this work, we propose a novel framework that integrates large language models (LLMs) with an RL-based dialogue manager for open-ended dialogue with a specific goal. By leveraging hierarchical reinforcement learning to model the structured phases of dialogue and employ meta-learning to enhance adaptability across diverse user profiles, our approach enhances adaptability and efficiency, enabling the system to learn from limited data, transition fluidly between dialogue phases, and personalize responses to heterogeneous patient needs. We apply our framework to Motivational Interviews, aiming to foster behavior change, and demonstrate that the proposed dialogue manager outperforms a state-of-the-art LLM baseline in terms of reward, showing a potential benefit of conditioning LLMs to create open-ended dialogue systems with specific goals.

Related papers

• Enhancing User Engagement in Socially-Driven Dialogue through Interactive LLM Alignments [36.632855175566746]
  Enhancing user engagement through interactions plays an essential role in socially-driven dialogues.<n>We enable interactive LLMs to learn user engagement by leveraging signals from the future development of conversations.<n>Experiments conducted on two socially-driven dialogue scenarios demonstrate that our method effectively enhances user engagement in interactive LLMs.
  arXiv  Detail & Related papers  (2025-06-26T17:26:17Z)
• Training Dialogue Systems by AI Feedback for Improving Overall Dialogue Impression [9.005722141359675]
  This study prepared reward models corresponding to 12 metrics related to the impression of the entire dialogue for evaluating dialogue responses.<n>We tuned our dialogue models using the reward model signals as feedback to improve the impression of the system.
  arXiv  Detail & Related papers  (2025-01-22T08:14:51Z)
• Data Augmentation Integrating Dialogue Flow and Style to Adapt Spoken Dialogue Systems to Low-Resource User Groups [1.7725414095035827]
  This study addresses the interaction challenges encountered by spoken dialogue systems (SDSs) when engaging with users who exhibit distinct conversational behaviors. We propose a novel data augmentation framework to enhance SDS performance for user groups with limited resources.
  arXiv  Detail & Related papers  (2024-08-20T03:33:04Z)
• Zero-Shot Goal-Directed Dialogue via RL on Imagined Conversations [70.7884839812069]
  Large language models (LLMs) have emerged as powerful and general solutions to many natural language tasks. However, many of the most important applications of language generation are interactive, where an agent has to talk to a person to reach a desired outcome. In this work, we explore a new method for adapting LLMs with RL for such goal-directed dialogue.
  arXiv  Detail & Related papers  (2023-11-09T18:45:16Z)
• Plug-and-Play Policy Planner for Large Language Model Powered Dialogue Agents [121.46051697742608]
  We introduce a new dialogue policy planning paradigm to strategize dialogue problems with a tunable language model plug-in named PPDPP. Specifically, we develop a novel training framework to facilitate supervised fine-tuning over available human-annotated data. PPDPP consistently and substantially outperforms existing approaches on three different proactive dialogue applications.
  arXiv  Detail & Related papers  (2023-11-01T03:20:16Z)
• Self-Explanation Prompting Improves Dialogue Understanding in Large Language Models [52.24756457516834]
  We propose a novel "Self-Explanation" prompting strategy to enhance the comprehension abilities of Large Language Models (LLMs) This task-agnostic approach requires the model to analyze each dialogue utterance before task execution, thereby improving performance across various dialogue-centric tasks. Experimental results from six benchmark datasets confirm that our method consistently outperforms other zero-shot prompts and matches or exceeds the efficacy of few-shot prompts.
  arXiv  Detail & Related papers  (2023-09-22T15:41:34Z)
• Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaboration [72.04629217161656]
  This work focuses on three aspects of proactive dialogue systems: clarification, target-guided, and non-collaborative dialogues. To trigger the proactivity of LLMs, we propose the Proactive Chain-of-Thought prompting scheme.
  arXiv  Detail & Related papers  (2023-05-23T02:49:35Z)
• A Mixture-of-Expert Approach to RL-based Dialogue Management [56.08449336469477]
  We use reinforcement learning to develop a dialogue agent that avoids being short-sighted (outputting generic utterances) and maximizes overall user satisfaction. Most existing RL approaches to DM train the agent at the word-level, and thus, have to deal with aly complex action space even for a medium-size vocabulary. We develop a RL-based DM using a novel mixture of expert language model (MoE-LM) that consists of (i) a LM capable of learning diverse semantics for conversation histories, (ii) a number of specialized LMs (or experts) capable of generating utterances corresponding to a
  arXiv  Detail & Related papers  (2022-05-31T19:00:41Z)
• Variational Hierarchical Dialog Autoencoder for Dialog State Tracking Data Augmentation [59.174903564894954]
  In this work, we extend this approach to the task of dialog state tracking for goal-oriented dialogs. We propose the Variational Hierarchical Dialog Autoencoder (VHDA) for modeling the complete aspects of goal-oriented dialogs. Experiments on various dialog datasets show that our model improves the downstream dialog trackers' robustness via generative data augmentation.
  arXiv  Detail & Related papers  (2020-01-23T15:34:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.