Persona-Aware Alignment Framework for Personalized Dialogue Generation

• URL: http://arxiv.org/abs/2511.10215v1
• Date: Fri, 14 Nov 2025 01:39:46 GMT
• Title: Persona-Aware Alignment Framework for Personalized Dialogue Generation
• Authors: Guanrong Li, Xinyu Liu, Zhen Wu, Xinyu Dai,
• Abstract summary: We propose a novel Persona-Aware Alignment Framework (PAL), which treats persona alignment as the training objective of dialogue generation.<n> PAL employs a two-stage training method including Persona-aware Learning and Persona Alignment, equipped with an easy-to-use inference strategy Select then Generate.<n>We demonstrate that our framework outperforms many state-of-the-art personalized dialogue methods and large language models.
• Score: 22.612334742745492
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Personalized dialogue generation aims to leverage persona profiles and dialogue history to generate persona-relevant and consistent responses. Mainstream models typically rely on token-level language model training with persona dialogue data, such as Next Token Prediction, to implicitly achieve personalization, making these methods tend to neglect the given personas and generate generic responses. To address this issue, we propose a novel Persona-Aware Alignment Framework (PAL), which directly treats persona alignment as the training objective of dialogue generation. Specifically, PAL employs a two-stage training method including Persona-aware Learning and Persona Alignment, equipped with an easy-to-use inference strategy Select then Generate, to improve persona sensitivity and generate more persona-relevant responses at the semantics level. Through extensive experiments, we demonstrate that our framework outperforms many state-of-the-art personalized dialogue methods and large language models.

Related papers

• Aligning Spoken Dialogue Models from User Interactions [55.192134724622235]
  We propose a novel preference alignment framework to improve spoken dialogue models on realtime conversations from user interactions.<n>We create a dataset of more than 150,000 preference pairs from raw multi-turn speech conversations annotated with AI feedback.<n>Our findings shed light on the importance of a well-calibrated balance among various dynamics, crucial for natural real-time speech dialogue systems.
  arXiv  Detail & Related papers  (2025-06-26T16:45:20Z)
• From Persona to Person: Enhancing the Naturalness with Multiple Discourse Relations Graph Learning in Personalized Dialogue Generation [11.442761234901289]
  We propose MUDI ($textbfMu$ltiple $textbfDi$scourse Relations Graph Learning) for personalized dialogue generation.<n>We utilize a Large Language Model to assist in annotating discourse relations and to transform dialogue data into structured dialogue graphs.<n>Our experiments demonstrate significant improvements in the quality of personalized responses, thus resembling human-like dialogue exchanges.
  arXiv  Detail & Related papers  (2025-06-13T08:12:52Z)
• A Personalized Conversational Benchmark: Towards Simulating Personalized Conversations [112.81207927088117]
  PersonaConvBench is a benchmark for evaluating personalized reasoning and generation in multi-turn conversations with large language models (LLMs)<n>We benchmark several commercial and open-source LLMs under a unified prompting setup and observe that incorporating personalized history yields substantial performance improvements.
  arXiv  Detail & Related papers  (2025-05-20T09:13:22Z)
• Dialogue Language Model with Large-Scale Persona Data Engineering [10.160626284195434]
  PPDS is an open-domain persona dialogue system that employs extensive generative pre-training on a persona dialogue dataset to enhance persona consistency.<n>We present a persona extraction model designed to autonomously and precisely generate vast persona dialogue datasets.<n>We also unveil a pioneering persona augmentation technique to address the invalid persona bias inherent in the constructed dataset.
  arXiv  Detail & Related papers  (2024-12-12T07:49:06Z)
• Dialogue Action Tokens: Steering Language Models in Goal-Directed Dialogue with a Multi-Turn Planner [51.77263363285369]
  We present an approach called Dialogue Action Tokens that adapts language model agents to plan goal-directed dialogues. The core idea is to treat each utterance as an action, thereby converting dialogues into games where existing approaches such as reinforcement learning can be applied.
  arXiv  Detail & Related papers  (2024-06-17T18:01:32Z)
• "In Dialogues We Learn": Towards Personalized Dialogue Without Pre-defined Profiles through In-Dialogue Learning [37.307408706864514]
  In-Dialogue Learning (IDL) is a fine-tuning framework that enhances the ability of pre-trained large language models to leverage dialogue history to characterize persona. Our experiments on three datasets demonstrate that IDL brings substantial improvements, with BLEU and ROUGE scores increasing by up to 200% and 247%, respectively.
  arXiv  Detail & Related papers  (2024-03-05T16:43:03Z)
• Personalized Language Modeling from Personalized Human Feedback [45.16986573937782]
  Personalized large language models (LLMs) are designed to tailor responses to individual user preferences.<n>We propose Personalized-RLHF, an efficient framework that utilizes a lightweight user model to capture individual user preferences.<n>We show that personalized LLMs trained using P-RLHF generate responses that are more closely aligned with individual user preferences.
  arXiv  Detail & Related papers  (2024-02-06T04:18:58Z)
• Using Natural Language Inference to Improve Persona Extraction from Dialogue in a New Domain [44.05974724495336]
  We introduce a natural language inference method for adapting a trained persona extraction model to a new setting. Our method returns higher-quality extracted persona and requires less human annotation.
  arXiv  Detail & Related papers  (2024-01-12T18:25:03Z)
• Less is More: Learning to Refine Dialogue History for Personalized Dialogue Generation [57.73547958927826]
  We propose to refine the user dialogue history on a large scale, based on which we can handle more dialogue history and obtain more accurate persona information. Specifically, we design an MSP model which consists of three personal information refiners and a personalized response generator.
  arXiv  Detail & Related papers  (2022-04-18T02:02:56Z)
• Learning to Predict Persona Information forDialogue Personalization without Explicit Persona Description [10.17868476063421]
  We propose a novel approach that learns to predict persona information based on the dialogue history to personalize the dialogue agent. Experimental results on the PersonaChat dataset show that the proposed method can improve the consistency of generated responses. A trained persona prediction model can be successfully transferred to other datasets and help generate more relevant responses.
  arXiv  Detail & Related papers  (2021-11-30T03:19:24Z)
• Will I Sound Like Me? Improving Persona Consistency in Dialogues through Pragmatic Self-Consciousness [62.55060760615656]
  Recent models tackling consistency often train with additional Natural Language Inference (NLI) labels or attach trained extra modules to the generative agent for maintaining consistency. Inspired by social cognition and pragmatics, we endow existing dialogue agents with public self-consciousness on the fly through an imaginary listener. Our approach, based on the Rational Speech Acts framework, can enforce dialogue agents to refrain from uttering contradiction.
  arXiv  Detail & Related papers  (2020-04-13T08:16:16Z)
• A Neural Topical Expansion Framework for Unstructured Persona-oriented Dialogue Generation [52.743311026230714]
  Persona Exploration and Exploitation (PEE) is able to extend the predefined user persona description with semantically correlated content. PEE consists of two main modules: persona exploration and persona exploitation. Our approach outperforms state-of-the-art baselines in terms of both automatic and human evaluations.
  arXiv  Detail & Related papers  (2020-02-06T08:24:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.