ChatSOP: An SOP-Guided MCTS Planning Framework for Controllable LLM Dialogue Agents

• URL: http://arxiv.org/abs/2407.03884v3
• Date: Sat, 22 Feb 2025 00:11:18 GMT
• Title: ChatSOP: An SOP-Guided MCTS Planning Framework for Controllable LLM Dialogue Agents
• Authors: Zhigen Li, Jianxiang Peng, Yanmeng Wang, Yong Cao, Tianhao Shen, Minghui Zhang, Linxi Su, Shang Wu, Yihang Wu, Yuqian Wang, Ye Wang, Wei Hu, Jianfeng Li, Shaojun Wang, Jing Xiao, Deyi Xiong,
• Abstract summary: We propose SOP-guided Monte Carlo Tree Search (MCTS) planning framework to enhance controllability of dialogue agents.<n>To enable this, we curate a dataset comprising SOP-annotated multi-scenario dialogues, generated using a semi-automated role-playing system with GPT-4o.<n>We also propose a novel method that integrates Chain of Thought reasoning with supervised fine-tuning for SOP prediction.
• Score: 52.7201882529976
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dialogue agents powered by Large Language Models (LLMs) show superior performance in various tasks. Despite the better user understanding and human-like responses, their lack of controllability remains a key challenge, often leading to unfocused conversations or task failure. To address this, we introduce Standard Operating Procedure (SOP) to regulate dialogue flow. Specifically, we propose ChatSOP, a novel SOP-guided Monte Carlo Tree Search (MCTS) planning framework designed to enhance the controllability of LLM-driven dialogue agents. To enable this, we curate a dataset comprising SOP-annotated multi-scenario dialogues, generated using a semi-automated role-playing system with GPT-4o and validated through strict manual quality control. Additionally, we propose a novel method that integrates Chain of Thought reasoning with supervised fine-tuning for SOP prediction and utilizes SOP-guided Monte Carlo Tree Search for optimal action planning during dialogues. Experimental results demonstrate the effectiveness of our method, such as achieving a 27.95% improvement in action accuracy compared to baseline models based on GPT-3.5 and also showing notable gains for open-source models. Dataset and codes are publicly available.

Related papers

• SAPIENT: Mastering Multi-turn Conversational Recommendation with Strategic Planning and Monte Carlo Tree Search [5.079888940901933]
  Existing methods train Reinforcement Learning-based agent with greedy action selection or sampling strategy. We present a novel Monte Carlo Tree Search (MCTS)-based CRS framework SAPIENT. SAPIENT consists of a conversational agent (S-agent) and a conversational planner (S-planner)
  arXiv  Detail & Related papers  (2024-10-12T16:21:33Z)
• Unsupervised Extraction of Dialogue Policies from Conversations [3.102576158218633]
  We show how Large Language Models can be instrumental in extracting dialogue policies from datasets. We then propose a novel method for generating dialogue policies utilizing a controllable and interpretable graph-based methodology.
  arXiv  Detail & Related papers  (2024-06-21T14:57:25Z)
• Learning to Clarify: Multi-turn Conversations with Action-Based Contrastive Self-Training [33.57497419019826]
  Action-Based Contrastive Self-Training allows for sample-efficient dialogue policy learning in multi-turn conversation. ACT demonstrates substantial conversation modeling improvements over standard approaches to supervised fine-tuning and DPO.
  arXiv  Detail & Related papers  (2024-05-31T22:44:48Z)
• Many Hands Make Light Work: Task-Oriented Dialogue System with Module-Based Mixture-of-Experts [9.129081545049992]
  Task-oriented dialogue systems have greatly benefited from pre-trained language models (PLMs) We propose Soft Mixture-of-Expert Task-Oriented Dialogue system (SMETOD) SMETOD leverages an ensemble of Mixture-of-Experts (MoEs) to excel at subproblems and generate specialized outputs for task-oriented dialogues. We extensively evaluate our model on three benchmark functionalities: intent prediction, dialogue state tracking, and dialogue response generation.
  arXiv  Detail & Related papers  (2024-05-16T01:02:09Z)
• Unsupervised End-to-End Task-Oriented Dialogue with LLMs: The Power of the Noisy Channel [9.082443585886127]
  Training task-oriented dialogue systems typically require turn-level annotations for interacting with their APIs. Unlabeled data and a schema definition are sufficient for building a working task-oriented dialogue system, completely unsupervised. We propose an innovative approach using expectation-maximization (EM) that infers turn-level annotations as latent variables.
  arXiv  Detail & Related papers  (2024-04-23T16:51:26Z)
• Large Language Models as Zero-shot Dialogue State Tracker through Function Calling [42.00097476584174]
  We propose a novel approach for solving dialogue state tracking with large language models (LLMs) through function calling. This method improves zero-shot DST, allowing adaptation to diverse domains without extensive data collection or model tuning. We show that our approach achieves exceptional performance with both modestly sized open-source and also proprietary LLMs.
  arXiv  Detail & Related papers  (2024-02-16T06:13:18Z)
• Zero-Shot Goal-Directed Dialogue via RL on Imagined Conversations [70.7884839812069]
  Large language models (LLMs) have emerged as powerful and general solutions to many natural language tasks. However, many of the most important applications of language generation are interactive, where an agent has to talk to a person to reach a desired outcome. In this work, we explore a new method for adapting LLMs with RL for such goal-directed dialogue.
  arXiv  Detail & Related papers  (2023-11-09T18:45:16Z)
• Plug-and-Play Policy Planner for Large Language Model Powered Dialogue Agents [121.46051697742608]
  We introduce a new dialogue policy planning paradigm to strategize dialogue problems with a tunable language model plug-in named PPDPP. Specifically, we develop a novel training framework to facilitate supervised fine-tuning over available human-annotated data. PPDPP consistently and substantially outperforms existing approaches on three different proactive dialogue applications.
  arXiv  Detail & Related papers  (2023-11-01T03:20:16Z)
• Symbolic Planning and Code Generation for Grounded Dialogue [78.48668501764385]
  Large language models (LLMs) excel at processing and generating both text and code. We present a modular and interpretable grounded dialogue system that addresses shortcomings by composing LLMs with a symbolic planner and grounded code execution. Our system substantially outperforms the previous state-of-the-art, including improving task success in human evaluations from 56% to 69% in the most challenging setting.
  arXiv  Detail & Related papers  (2023-10-26T04:22:23Z)
• Self-Explanation Prompting Improves Dialogue Understanding in Large Language Models [52.24756457516834]
  We propose a novel "Self-Explanation" prompting strategy to enhance the comprehension abilities of Large Language Models (LLMs) This task-agnostic approach requires the model to analyze each dialogue utterance before task execution, thereby improving performance across various dialogue-centric tasks. Experimental results from six benchmark datasets confirm that our method consistently outperforms other zero-shot prompts and matches or exceeds the efficacy of few-shot prompts.
  arXiv  Detail & Related papers  (2023-09-22T15:41:34Z)
• Unlocking the Potential of User Feedback: Leveraging Large Language Model as User Simulator to Enhance Dialogue System [65.93577256431125]
  We propose an alternative approach called User-Guided Response Optimization (UGRO) to combine it with a smaller task-oriented dialogue model. This approach uses LLM as annotation-free user simulator to assess dialogue responses, combining them with smaller fine-tuned end-to-end TOD models. Our approach outperforms previous state-of-the-art (SOTA) results.
  arXiv  Detail & Related papers  (2023-06-16T13:04:56Z)
• Leveraging Explicit Procedural Instructions for Data-Efficient Action Prediction [5.448684866061922]
  Task-oriented dialogues often require agents to enact complex, multi-step procedures in order to meet user requests. Large language models have found success automating these dialogues in constrained environments, but their widespread deployment is limited by the substantial quantities of task-specific data required for training. This paper presents a data-efficient solution to constructing dialogue systems, leveraging explicit instructions derived from agent guidelines.
  arXiv  Detail & Related papers  (2023-06-06T18:42:08Z)
• Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaboration [72.04629217161656]
  This work focuses on three aspects of proactive dialogue systems: clarification, target-guided, and non-collaborative dialogues. To trigger the proactivity of LLMs, we propose the Proactive Chain-of-Thought prompting scheme.
  arXiv  Detail & Related papers  (2023-05-23T02:49:35Z)
• Controllable Mixed-Initiative Dialogue Generation through Prompting [50.03458333265885]
  Mixed-initiative dialogue tasks involve repeated exchanges of information and conversational control. Agents gain control by generating responses that follow particular dialogue intents or strategies, prescribed by a policy planner. Standard approach has been fine-tuning pre-trained language models to perform generation conditioned on these intents. We instead prompt large language models as a drop-in replacement to fine-tuning on conditional generation.
  arXiv  Detail & Related papers  (2023-05-06T23:11:25Z)
• In-Context Learning for Few-Shot Dialogue State Tracking [55.91832381893181]
  We propose an in-context (IC) learning framework for few-shot dialogue state tracking (DST) A large pre-trained language model (LM) takes a test instance and a few annotated examples as input, and directly decodes the dialogue states without any parameter updates. This makes the LM more flexible and scalable compared to prior few-shot DST work when adapting to new domains and scenarios.
  arXiv  Detail & Related papers  (2022-03-16T11:58:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.