Understanding Generalization in Role-Playing Models via Information Theory

• URL: http://arxiv.org/abs/2512.17270v1
• Date: Fri, 19 Dec 2025 06:37:44 GMT
• Title: Understanding Generalization in Role-Playing Models via Information Theory
• Abstract summary: We introduce an information-theoretic metric, named reasoning-based effective mutual information difference (R-EMID), to measure RPM performance degradation.<n>We also propose a co-evolving reinforcement learning framework to adaptively model the connection among user, character, and dialogue context.
• Score: 78.78610491091096
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Role-playing models (RPMs) are widely used in real-world applications but underperform when deployed in the wild. This degradation can be attributed to distribution shifts, including user, character, and dialogue compositional shifts. Existing methods like LLM-as-a-judge fall short in providing a fine-grained diagnosis of how these shifts affect RPM generalization, and thus there lack formal frameworks to characterize RPM generalization behaviors. To bridge these gaps, we introduce an information-theoretic metric, named reasoning-based effective mutual information difference (R-EMID), to measure RPM performance degradation in an interpretable way. We also derive an upper bound on R-EMID to predict the worst-case generalization performance of RPMs and theoretically reveal how various shifts contribute to the RPM performance degradation. Moreover, we propose a co-evolving reinforcement learning framework to adaptively model the connection among user, character, and dialogue context and thus enhance the estimation of dialogue response generation probability, which is critical for calculating R-EMID. Finally, we evaluate the generalization performance of various RPMs using R-EMID, finding that user shift poses the highest risk among all shifts and reinforcement learning is the most effective approach for enhancing RPM generalization.

Related papers

• CRPO: Character-centric Group Relative Policy Optimization for Role-aware Reasoning in Role-playing Agents [53.765941044015854]
  We propose Character-Centric Group Relative Policy Optimization (CRPO), a framework designed to realign objectives with the role-playing task.<n>CRPO improves character distinctiveness through three mechanisms: decoupling task logic from stylistic rewards to resolve gradient conflicts, dynamically adapting optimization constraints based on character complexity, and utilizing generic responses as negative baselines to prevent the model from reverting to a common distribution.
  arXiv  Detail & Related papers  (2026-05-25T07:15:38Z)
• Metacognition as Reward: Reinforcing LLM Reasoning via Knowledge and Regulation Signals [75.25256166997414]
  We introduce Metacognition-as-Reward (MaR), a metacognition-inspired RL framework.<n>MaR guides reasoning through two general process dimensions.<n>MaR consistently improves model performance, achieving up to a 7.7% gain over the base model.
  arXiv  Detail & Related papers  (2026-05-22T08:54:37Z)
• Evaluating Reward Model Generalization via Pairwise Maximum Discrepancy Competitions [24.01200309422524]
  Pairwise Maximum Discrepancy Competition (PMDC) is a dynamic and annotation-efficient framework for evaluating RM generalization.<n>PMDC actively selects prompt--response pairs that maximize disagreement between two RMs.<n>We apply PMDC to re-evaluate 10 representative RMs and observe substantial rank reshuffling compared with conventional benchmarks.
  arXiv  Detail & Related papers  (2026-01-05T15:14:21Z)
• Mixture of Ranks with Degradation-Aware Routing for One-Step Real-World Image Super-Resolution [76.66229730098759]
  In real-world image super-resolution (Real-ISR), existing approaches mainly rely on fine-tuning pre-trained diffusion models.<n>We propose a Mixture-of-Ranks (MoR) architecture for single-step image super-resolution.<n>We introduce a fine-grained expert partitioning strategy that treats each rank in LoRA as an independent expert.
  arXiv  Detail & Related papers  (2025-11-20T04:11:44Z)
• Meta-Policy Reflexion: Reusable Reflective Memory and Rule Admissibility for Resource-Efficient LLM Agent [6.300669721057781]
  Meta-Policy Reflexion (MPR) is a framework that consolidates LLM-generated reflections into a structured, predicate-like Meta-Policy Memory (MPM)<n>MPR externalizes reusable corrective knowledge without model weight updates, enforces domain constraints to reduce unsafe or invalid actions, and retains the adaptability of language-based reflection.<n> Empirical results reported in the supplied material indicate consistent gains in execution accuracy and robustness when compared to Reflexion baselines; rule admissibility further improves stability.
  arXiv  Detail & Related papers  (2025-09-04T08:18:39Z)
• Attributing Response to Context: A Jensen-Shannon Divergence Driven Mechanistic Study of Context Attribution in Retrieval-Augmented Generation [52.3707788779464]
  We introduce a novel Jensen-Shannon Divergence driven method to Attribute Response to Context (ARC-JSD)<n>ARC-JSD enables efficient and accurate identification of essential context sentences without additional fine-tuning, gradient-calculation or surrogate modelling.<n> Evaluations on a wide range of RAG benchmarks, such as TyDi QA, Hotpot QA, and Musique, using instruction-tuned LLMs in different scales demonstrate superior accuracy and significant computational efficiency improvements.
  arXiv  Detail & Related papers  (2025-05-22T09:04:03Z)
• Sensitivity Meets Sparsity: The Impact of Extremely Sparse Parameter Patterns on Theory-of-Mind of Large Language Models [55.46269953415811]
  We identify ToM-sensitive parameters and show that perturbing as little as 0.001% of these parameters significantly degrades ToM performance.<n>Our results have implications for enhancing model alignment, mitigating biases, and improving AI systems designed for human interaction.
  arXiv  Detail & Related papers  (2025-04-05T17:45:42Z)
• An In-depth Investigation of Sparse Rate Reduction in Transformer-like Models [32.04194224236952]
  We propose an information-theoretic objective function called Sparse Rate Reduction (SRR) We show that SRR has a positive correlation coefficient and outperforms other baseline measures, such as path-norm and sharpness-based ones. We show that generalization can be improved using SRR as regularization on benchmark image classification datasets.
  arXiv  Detail & Related papers  (2024-11-26T07:44:57Z)
• Out-of-distribution robustness for multivariate analysis via causal regularisation [4.487663958743944]
  We propose a regularisation strategy rooted in causality that ensures robustness against distribution shifts.<n>Building upon the anchor regression framework, we demonstrate how incorporating a straightforward regularisation term into the loss function of classical algorithms.<n>Our framework allows users to efficiently verify the compatibility of a loss function with the regularisation strategy.
  arXiv  Detail & Related papers  (2024-03-04T09:21:10Z)
• ASR: Attention-alike Structural Re-parameterization [53.019657810468026]
  We propose a simple-yet-effective attention-alike structural re- parameterization (ASR) that allows us to achieve SRP for a given network while enjoying the effectiveness of the attention mechanism. In this paper, we conduct extensive experiments from a statistical perspective and discover an interesting phenomenon Stripe Observation, which reveals that channel attention values quickly approach some constant vectors during training.
  arXiv  Detail & Related papers  (2023-04-13T08:52:34Z)
• Provable Generalization of Overparameterized Meta-learning Trained with SGD [62.892930625034374]
  We study the generalization of a widely used meta-learning approach, Model-Agnostic Meta-Learning (MAML) We provide both upper and lower bounds for the excess risk of MAML, which captures how SGD dynamics affect these generalization bounds. Our theoretical findings are further validated by experiments.
  arXiv  Detail & Related papers  (2022-06-18T07:22:57Z)
• Multi-Label Contrastive Learning for Abstract Visual Reasoning [0.0]
  State-of-the-art systems solving Raven's Progressive Matrices rely on massive pattern-based training and exploiting biases in the dataset. Humans concentrate on identification of the rules / concepts underlying the RPM (or generally a visual reasoning task) to be solved. We propose a new sparse rule encoding scheme for RPMs which, besides the new training algorithm, is the key factor contributing to the state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-12-03T14:18:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.