Language Models Don't Know What You Want: Evaluating Personalization in Deep Research Needs Real Users

• URL: http://arxiv.org/abs/2603.16120v1
• Date: Tue, 17 Mar 2026 04:59:32 GMT
• Title: Language Models Don't Know What You Want: Evaluating Personalization in Deep Research Needs Real Users
• Abstract summary: Deep Research (DR) tools can synthesize scientific papers to answer researchers' queries, but lack understanding of their users.<n>We develop a personalized DR tool that infers a profile of a user's research interests and proposes personalized actions for a user's input query.<n>We reveal nine nuanced errors of personalized DR undetectable by our LLM judges, and we study qualitative feedback to form lessons for future DR design.
• Score: 48.61084675902644
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep Research (DR) tools (e.g. OpenAI DR) help researchers cope with ballooning publishing counts. Such tools can synthesize scientific papers to answer researchers' queries, but lack understanding of their users. We change that in MyScholarQA (MySQA), a personalized DR tool that: 1) infers a profile of a user's research interests; 2) proposes personalized actions for a user's input query; and 3) writes a multi-section report for the query that follows user-approved actions. We first test MySQA with NLP's standard protocol: we design a benchmark of synthetic users and LLM judges, where MySQA beats baselines in citation metrics and personalized action-following. However, we suspect this process does not cover all aspects of personalized DR users value, so we interview users in an online version of MySQA to unmask them. We reveal nine nuanced errors of personalized DR undetectable by our LLM judges, and we study qualitative feedback to form lessons for future DR design. In all, we argue for a pillar of personalization that easy-to-use LLM judges can lead NLP to overlook: real progress in personalization is only possible with real users.

Related papers

• DRACULA: Hunting for the Actions Users Want Deep Research Agents to Execute [66.25314396197075]
  We collect DRACULA, the first dataset with user feedback on intermediate actions for Scientific Deep Research (DR) agents.<n>Our users select actions they prefer, then judge whether an output report applied their selections successfully.<n>We study the predictability of user-preferred actions via simulation-how well LLMs predict the actions users select-a step toward learning to generate useful actions.
  arXiv  Detail & Related papers  (2026-04-26T17:32:39Z)
• Towards Realistic Personalization: Evaluating Long-Horizon Preference Following in Personalized User-LLM Interactions [50.70965714314064]
  Large Language Models (LLMs) are increasingly serving as personal assistants, where users share complex and diverse preferences over extended interactions.<n>This work proposes RealPref, a benchmark for evaluating realistic preference-following in personalized user-LLM interactions.
  arXiv  Detail & Related papers  (2026-03-04T15:42:43Z)
• Learning to Reason for Multi-Step Retrieval of Personal Context in Personalized Question Answering [39.08300602619814]
  Personalization in Question Answering (QA) requires answers that are both accurate and aligned with users' background, preferences, and historical context.<n>We propose PR2, a reinforcement learning framework that integrates reasoning and retrieval from personal context for personalization.
  arXiv  Detail & Related papers  (2026-02-22T19:43:43Z)
• MLLM as a UI Judge: Benchmarking Multimodal LLMs for Predicting Human Perception of User Interfaces [97.62557395494962]
  We use crowdsourcing to benchmark GPT-4o, Claude, and Llama across 30 interfaces.<n>Our results show that MLLMs approximate human preferences on some dimensions but diverge on others.
  arXiv  Detail & Related papers  (2025-10-09T20:00:41Z)
• Few-Shot and Training-Free Review Generation via Conversational Prompting [2.0305676256390934]
  Real-world applications often face few-shot and training-free situations.<n>We propose Conversational Prompting, a lightweight method that reformulates user reviews as multi-turn conversations.
  arXiv  Detail & Related papers  (2025-09-25T06:36:08Z)
• PersonaFeedback: A Large-scale Human-annotated Benchmark For Personalization [25.45861816665351]
  We introduce PersonaFeedback, a new benchmark that directly evaluates LLMs' ability to provide personalized responses.<n>Unlike existing benchmarks that require models to infer implicit user personas from historical interactions, PersonaFeedback decouples persona inference from personalization.<n> PersonaFeedback consists of 8298 human-annotated test cases, which are categorized into easy, medium, and hard tiers.
  arXiv  Detail & Related papers  (2025-06-15T17:19:19Z)
• Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scale [53.059480071818136]
  Large Language Models (LLMs) have emerged as personalized assistants for users across a wide range of tasks.<n> PERSONAMEM features curated user profiles with over 180 simulated user-LLM interaction histories.<n>We evaluate LLM chatbots' ability to identify the most suitable response according to the current state of the user's profile.
  arXiv  Detail & Related papers  (2025-04-19T08:16:10Z)
• Step-Back Profiling: Distilling User History for Personalized Scientific Writing [50.481041470669766]
  Large language models (LLM) excel at a variety of natural language processing tasks, yet they struggle to generate personalized content for individuals. We introduce STEP-BACK PROFILING to personalize LLMs by distilling user history into concise profiles. Our approach outperforms the baselines by up to 3.6 points on the general personalization benchmark.
  arXiv  Detail & Related papers  (2024-06-20T12:58:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.