ReLay: Personalized LLM-Generated Plain-Language Summaries for Better Understanding, but at What Cost?

• URL: http://arxiv.org/abs/2605.00468v1
• Date: Fri, 01 May 2026 07:11:12 GMT
• Title: ReLay: Personalized LLM-Generated Plain-Language Summaries for Better Understanding, but at What Cost?
• Abstract summary: Large language models (LLMs) offer new opportunities for personalizing Plain Language Summaries (PLS)<n>It remains unclear whether personalization helps, which strategies are most effective, and how to balance personalization with safety.<n>We introduce ReLay, a dataset of 300 participant--PLS pairs from 50 lay participants in both static (expert-written) and interactive settings.<n>ReLay includes user characteristics, health information needs, information-seeking behavior, comprehension outcomes, interaction logs, and quality ratings.
• Score: 54.44351107428039
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Plain Language Summaries (PLS) aim to make research accessible to lay readers, but they are typically written in a one-size-fits-all style that ignores differences in readers' information needs and comprehension. In health contexts, this limitation is particularly important because misunderstanding scientific information can affect real-world decisions. Large language models (LLMs) offer new opportunities for personalizing PLS, but it remains unclear whether personalization helps, which strategies are most effective, and how to balance personalization with safety. We introduce ReLay, a dataset of 300 participant--PLS pairs from 50 lay participants in both static (expert-written) and interactive (LLM-personalized) settings. ReLay includes user characteristics, health information needs, information-seeking behavior, comprehension outcomes, interaction logs, and quality ratings. We use ReLay to evaluate five LLMs across two personalization methods. Personalization improves comprehension and perceived quality, but it also raises the risk of reinforcing user biases and introducing hallucinations, revealing a trade-off between personalization and safety. These findings highlight the need for personalization methods that are both effective and trustworthy for diverse lay audiences.

Related papers

• Evaluating the Hidden Costs of Personalization in Large Language Models [55.07935163900691]
  We identify three emerging risks: (1) irrelevant personalization, where models reference personal information in unnecessary contexts; (2) preference narrowing, where models reinforce informational echo chambers; and (3) sycophantic bias, where models agree excessively with user opinions.<n>We propose PRISK, a dynamic evaluation framework with automated data generation and tailored metrics that uncovers systematic limitations in current AI assistants.
  arXiv  Detail & Related papers  (2026-08-28T20:05:12Z)
• Re-Centering Humans in LLM Personalization [15.480006171928435]
  We study the gap in personalization performance when using synthetic versus human data.<n>We collect human conversations (550 conversations) and judgments across three stages of personalization.<n>We find that learned reward models achieve only modest correlation with human ratings.
  arXiv  Detail & Related papers  (2026-06-04T18:04:47Z)
• Enhancing Debunking Effectiveness through LLM-based Personality Adaptation [41.32471203510647]
  This study proposes a novel methodology for generating personalized fake news debunking messages.<n>We use Large Language Models (LLMs) with persona-based inputs aligned to the Big Five personality traits: Extraversion, Agreeableness, Conscientiousness, Neuroticism, and Openness.<n>Our results show that personalized messages are generally seen as more persuasive than generic ones.
  arXiv  Detail & Related papers  (2026-03-10T11:44:17Z)
• Personality as Relational Infrastructure: User Perceptions of Personality-Trait-Infused LLM Messaging [0.6999740786886536]
  We show that personality-based personalisation in behaviour change systems may operate primarily through aggregate exposure rather than per-message.<n>In-situ longitudinal studies are needed to validate these findings in real-world contexts.
  arXiv  Detail & Related papers  (2026-02-06T10:47:47Z)
• PersonaDual: Balancing Personalization and Objectivity via Adaptive Reasoning [57.10486355722133]
  PersonaDual supports both general-purpose objective reasoning and personalized reasoning in a single model.<n>Experiments on objective and personalized benchmarks show that PersonaDual preserves the benefits of personalization while reducing interference.
  arXiv  Detail & Related papers  (2026-01-13T16:02:35Z)
• A Multilingual, Large-Scale Study of the Interplay between LLM Safeguards, Personalisation, and Disinformation [12.577461004484604]
  This study presents the first large-scale, multilingual analysis of persona-targeted disinformation generation by Large Language Models (LLMs)<n>We prompt eight state-of-the-art LLMs with 324 false narratives and 150 demographic personas across four languages--English, Russian, Portuguese, and Hindi--resulting in AI-TRAITS.<n>Results show that the use of even simple personalisation prompts significantly increases the likelihood of jailbreaks.
  arXiv  Detail & Related papers  (2025-10-14T21:10:50Z)
• Learning to summarize user information for personalized reinforcement learning from human feedback [19.859785715555013]
  Preference Learning Using Summarization (PLUS) uses reinforcement learning to learn to produce text-based summaries of each user's preferences.<n>Both the user-summarization model and reward model are trained simultaneously, creating an online co-adaptation loop.<n>We show that PLUS capture diverse aspects of user preferences, achieving a 11-77% improvement in reward model accuracy.
  arXiv  Detail & Related papers  (2025-07-17T23:48:51Z)
• Enhancing Personalized Multi-Turn Dialogue with Curiosity Reward [15.054368169735872]
  We propose leveraging a user model to incorporate a curiosity-based intrinsic reward into multi-turn RLHF.<n>This novel reward mechanism encourages the LLM agent to actively infer user traits by optimizing conversations to improve its user model's accuracy.<n>We demonstrate our method's effectiveness in two distinct domains: significantly improving personalization performance in a conversational recommendation task, and personalizing conversations for different learning styles in an educational setting.
  arXiv  Detail & Related papers  (2025-04-04T06:35:02Z)
• FSPO: Few-Shot Preference Optimization of Synthetic Preference Data in LLMs Elicits Effective Personalization to Real Users [111.56469697145519]
  We propose Few-Shot Preference Optimization, which reframes reward modeling as a meta-learning problem.<n>Under this framework, an LLM learns to quickly adapt to a user via a few labeled preferences from that user, constructing a personalized reward function for them.<n>We generate over 1M synthetic personalized preferences using publicly available LLMs.<n>We evaluate FSPO on personalized open-ended generation for up to 1,500 synthetic users across three domains: movie reviews, pedagogical adaptation based on educational background, and general question answering, along with a controlled human study.
  arXiv  Detail & Related papers  (2025-02-26T17:08:46Z)
• A Survey of Personalized Large Language Models: Progress and Future Directions [86.45576419251302]
  Large Language Models (LLMs) excel in handling general knowledge tasks, yet struggle with user-specific personalization.<n> Personalized Large Language Models (PLLMs) tackle these challenges by leveraging individual user data.<n>PLLMs can significantly enhance user satisfaction and have broad applications in conversational agents, systems, emotion recognition, medical assistants, and more.
  arXiv  Detail & Related papers  (2025-02-17T07:58:31Z)
• Personality Alignment of Large Language Models [30.710131188931317]
  Personality Alignment aims to align large language models with individual user preferences.<n>This dataset includes data from over 320,000 real subjects across multiple personality assessments.<n>We develop an activation intervention optimization method to efficiently align with individual behavioral preferences.<n>Our work paves the way for future AI systems to make decisions and reason in truly personality ways.
  arXiv  Detail & Related papers  (2024-08-21T17:09:00Z)
• PsyCoT: Psychological Questionnaire as Powerful Chain-of-Thought for Personality Detection [50.66968526809069]
  We propose a novel personality detection method, called PsyCoT, which mimics the way individuals complete psychological questionnaires in a multi-turn dialogue manner. Our experiments demonstrate that PsyCoT significantly improves the performance and robustness of GPT-3.5 in personality detection.
  arXiv  Detail & Related papers  (2023-10-31T08:23:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.