BEYOND DIALOGUE: A Profile-Dialogue Alignment Framework Towards General Role-Playing Language Model

• URL: http://arxiv.org/abs/2408.10903v5
• Date: Thu, 29 Aug 2024 02:38:05 GMT
• Title: BEYOND DIALOGUE: A Profile-Dialogue Alignment Framework Towards General Role-Playing Language Model
• Authors: Yeyong Yu, Runsheng Yu, Haojie Wei, Zhanqiu Zhang, Quan Qian,
• Abstract summary: The rapid advancement of large language models (LLMs) has revolutionized role-playing, enabling the development of general role-playing models. Current role-playing training has two significant issues: (I) Using a predefined role profile to prompt dialogue training for specific scenarios leads to inconsistencies and even conflicts between the dialogue and the profile, resulting in training biases. We propose a simple yet effective framework called DIALOGUE, designed to overcome these hurdles.
• Score: 12.617285298415013
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The rapid advancement of large language models (LLMs) has revolutionized role-playing, enabling the development of general role-playing models. However, current role-playing training has two significant issues: (I) Using a predefined role profile to prompt dialogue training for specific scenarios usually leads to inconsistencies and even conflicts between the dialogue and the profile, resulting in training biases. (II) The model learns to imitate the role based solely on the profile, neglecting profile-dialogue alignment at the sentence level. In this work, we propose a simple yet effective framework called BEYOND DIALOGUE, designed to overcome these hurdles. This framework innovatively introduces "beyond dialogue" tasks to align dialogue with profile traits based on each specific scenario, thereby eliminating biases during training. Furthermore, by adopting an innovative prompting mechanism that generates reasoning outcomes for training, the framework allows the model to achieve fine-grained alignment between profile and dialogue at the sentence level. The aforementioned methods are fully automated and low-cost. Additionally, the integration of automated dialogue and objective evaluation methods forms a comprehensive framework, paving the way for general role-playing. Experimental results demonstrate that our model excels in adhering to and reflecting various dimensions of role profiles, outperforming most proprietary general and specialized role-playing baselines. All code and datasets are available at https://github.com/yuyouyu32/BeyondDialogue.

Related papers

• PRODIGy: a PROfile-based DIalogue Generation dataset [12.566555088877871]
  We propose a new resource where each dialogue is aligned with all possible speaker representations such as communication style, biographies, and personality. This framework allows to test several baselines built using generative language models with several profile configurations.
  arXiv  Detail & Related papers  (2023-11-09T08:19:34Z)
• JoTR: A Joint Transformer and Reinforcement Learning Framework for Dialog Policy Learning [53.83063435640911]
  Dialogue policy learning (DPL) is a crucial component of dialogue modelling. We introduce a novel framework, JoTR, to generate flexible dialogue actions. Unlike traditional methods, JoTR formulates a word-level policy that allows for a more dynamic and adaptable dialogue action generation.
  arXiv  Detail & Related papers  (2023-09-01T03:19:53Z)
• Revisiting Conversation Discourse for Dialogue Disentanglement [88.3386821205896]
  We propose enhancing dialogue disentanglement by taking full advantage of the dialogue discourse characteristics. We develop a structure-aware framework to integrate the rich structural features for better modeling the conversational semantic context. Our work has great potential to facilitate broader multi-party multi-thread dialogue applications.
  arXiv  Detail & Related papers  (2023-06-06T19:17:47Z)
• Pre-training Multi-party Dialogue Models with Latent Discourse Inference [85.9683181507206]
  We pre-train a model that understands the discourse structure of multi-party dialogues, namely, to whom each utterance is replying. To fully utilize the unlabeled data, we propose to treat the discourse structures as latent variables, then jointly infer them and pre-train the discourse-aware model.
  arXiv  Detail & Related papers  (2023-05-24T14:06:27Z)
• DIONYSUS: A Pre-trained Model for Low-Resource Dialogue Summarization [127.714919036388]
  DIONYSUS is a pre-trained encoder-decoder model for summarizing dialogues in any new domain. Our experiments show that DIONYSUS outperforms existing methods on six datasets.
  arXiv  Detail & Related papers  (2022-12-20T06:21:21Z)
• Representation Learning for Conversational Data using Discourse Mutual Information Maximization [9.017156603976915]
  We argue that the structure-unaware word-by-word generation is not suitable for effective conversation modeling. We propose a structure-aware Mutual Information based loss-function DMI for training dialog-representation models. Our models show the most promising performance on the dialog evaluation task DailyDialog++, in both random and adversarial negative scenarios.
  arXiv  Detail & Related papers  (2021-12-04T13:17:07Z)
• GALAXY: A Generative Pre-trained Model for Task-Oriented Dialog with Semi-Supervised Learning and Explicit Policy Injection [36.77204909711832]
  We propose a novel pre-trained dialog model that explicitly learns dialog policy from limited labeled dialogs and large-scale unlabeled dialog corpora. Specifically, we introduce a dialog act prediction task for policy optimization during pre-training and employ a consistency regularization term to refine the learned representation. Empirical results show that GALAXY substantially improves the performance of task-oriented dialog systems.
  arXiv  Detail & Related papers  (2021-11-29T15:24:36Z)
• Structural Pre-training for Dialogue Comprehension [51.215629336320305]
  We present SPIDER, Structural Pre-traIned DialoguE Reader, to capture dialogue exclusive features. To simulate the dialogue-like features, we propose two training objectives in addition to the original LM objectives. Experimental results on widely used dialogue benchmarks verify the effectiveness of the newly introduced self-supervised tasks.
  arXiv  Detail & Related papers  (2021-05-23T15:16:54Z)
• RADDLE: An Evaluation Benchmark and Analysis Platform for Robust Task-oriented Dialog Systems [75.87418236410296]
  We introduce the RADDLE benchmark, a collection of corpora and tools for evaluating the performance of models across a diverse set of domains. RADDLE is designed to favor and encourage models with a strong generalization ability. We evaluate recent state-of-the-art systems based on pre-training and fine-tuning, and find that grounded pre-training on heterogeneous dialog corpora performs better than training a separate model per domain.
  arXiv  Detail & Related papers  (2020-12-29T08:58:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.