Personalized Deep Research: A User-Centric Framework, Dataset, and Hybrid Evaluation for Knowledge Discovery

• URL: http://arxiv.org/abs/2605.10530v1
• Date: Mon, 11 May 2026 13:14:54 GMT
• Title: Personalized Deep Research: A User-Centric Framework, Dataset, and Hybrid Evaluation for Knowledge Discovery
• Abstract summary: We introduce Personalized Deep Research (PDR), a framework that integrates dynamic user context into the core retrieval-reasoning loop.<n>PDR unifies user profile modeling with iterative query development, dual-stage (private/public) retrieval, and context-aware synthesis.<n>This allows the system to autonomously align research sub-goals with user intent and optimize the stopping criteria for evidence collection.
• Score: 36.57361069825463
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep Research agents driven by LLMs have automated the scholarly discovery pipeline, from planning and query formulation to iterative web exploration. Yet they remain constrained by a static, ``one-size-fits-all'' retrieval paradigm. Current systems fail to adaptively adjust the depth and breadth of exploration based on the user's existing expertise or latent interests, frequently resulting in reports that are either redundant for experts or overly dense for novices. To address this, we introduce Personalized Deep Research (PDR), a framework that integrates dynamic user context into the core retrieval-reasoning loop. Rather than treating personalization as a post-hoc formatting step, PDR unifies user profile modeling with iterative query development, dual-stage (private/public) retrieval, and context-aware synthesis. This allows the system to autonomously align research sub-goals with user intent and optimize the stopping criteria for evidence collection. To facilitate benchmarking, we release the PDR Dataset, covering four realistic user tasks, and propose a hybrid evaluation framework combining lexical metrics with LLM-based judgments to assess factual accuracy and personalization alignment. Experimental results against commercial baselines demonstrate that PDR significantly improves retrieval utility and report relevance, effectively bridging the gap between generic information retrieval and personalized knowledge acquisition. The resource is available to the public at https://github.com/Applied-Machine-Learning-Lab/SIGIR2026_PDR.

Related papers

• Think-to-Personalize: Unifying Reasoning and Retrieval for User-Centric Personalized Dense Retrieval [37.821022474615994]
  Think-to-Personalize (TTP) is a novel framework that unifies explicit user-centric intent reasoning with dense retrieval.<n>TTP significantly outperforms state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-08-19T12:28:20Z)
• Learning to Reason for Multi-Step Retrieval of Personal Context in Personalized Question Answering [39.08300602619814]
  Personalization in Question Answering (QA) requires answers that are both accurate and aligned with users' background, preferences, and historical context.<n>We propose PR2, a reinforcement learning framework that integrates reasoning and retrieval from personal context for personalization.
  arXiv  Detail & Related papers  (2026-02-22T19:43:43Z)
• Learning Query-Specific Rubrics from Human Preferences for DeepResearch Report Generation [80.12435680651488]
  We propose a pipeline to train human-preference-aligned query-specific rubric generators tailored for DeepResearch report generation.<n>We first construct a dataset of DeepResearch-style annotated queries with human preferences over paired reports, and train rubric generators via reinforcement learning.<n>We empirically show that our proposed rubric generators deliver more discriminative and better human-aligned supervision than existing rubric design strategies.
  arXiv  Detail & Related papers  (2026-02-03T15:09:56Z)
• Investigating LLM Variability in Personalized Conversational Information Retrieval [14.220276130333849]
  Mo et al. explored several strategies for incorporating Personal Textual Knowledge Bases (PTKB) into Large Language Models (LLMs)<n>We apply the original methods to a new TREC iKAT 2024 dataset and evaluate a diverse range of models, including Llama (1B-70B), Qwen-7B, GPT-4o-mini.<n>Our results show that human-selected PTKBs consistently enhance retrieval performance, while LLM-based selection methods do not reliably outperform manual choices.
  arXiv  Detail & Related papers  (2025-10-04T12:13:19Z)
• DRBench: A Realistic Benchmark for Enterprise Deep Research [81.49694432639406]
  DRBench is a benchmark for evaluating AI agents on complex, open-ended deep research tasks in enterprise settings.<n>We release 15 deep research tasks across 10 domains, such as Sales, Cybersecurity, and Compliance.
  arXiv  Detail & Related papers  (2025-09-30T18:47:20Z)
• Towards Personalized Deep Research: Benchmarks and Evaluations [56.581105664044436]
  We introduce Personalized Deep Research Bench, the first benchmark for evaluating personalization in Deep Research Agents (DRAs)<n>It pairs 50 diverse research tasks with 25 authentic user profiles that combine structured persona attributes with dynamic real-world contexts, yielding 250 realistic user-task queries.<n>Our experiments on a range of systems highlight current capabilities and limitations in handling personalized deep research.
  arXiv  Detail & Related papers  (2025-09-29T17:39:17Z)
• BESPOKE: Benchmark for Search-Augmented Large Language Model Personalization via Diagnostic Feedback [9.980170820190093]
  We propose BESPOKE, the realistic benchmark for evaluating personalization in search-augmented large language models.<n>BESPOKE is designed to be both realistic, by collecting authentic chat and search histories directly from humans.<n>We conduct systematic analyses that reveal key requirements for effective personalization in information-seeking tasks.
  arXiv  Detail & Related papers  (2025-09-25T12:53:07Z)
• Intent-Aware Neural Query Reformulation for Behavior-Aligned Product Search [0.0]
  This work introduces a robust data pipeline designed to mine and analyze large-scale buyer query logs.<n>The pipeline systematically captures patterns indicative of latent purchase intent, enabling the construction of a high-fidelity, intent-rich dataset.<n>Our findings highlight the value of intent-centric modeling in bridging the gap between sparse user inputs and complex product discovery goals.
  arXiv  Detail & Related papers  (2025-07-29T20:20:07Z)
• Benchmarking Deep Search over Heterogeneous Enterprise Data [73.55304268238474]
  We present a new benchmark for evaluating a form of retrieval-augmented generation (RAG)<n>RAG requires source-aware, multi-hop reasoning over diverse, sparsed, but related sources.<n>We build it using a synthetic data pipeline that simulates business across product planning, development, and support stages.
  arXiv  Detail & Related papers  (2025-06-29T08:34:59Z)
• InfoDeepSeek: Benchmarking Agentic Information Seeking for Retrieval-Augmented Generation [63.55258191625131]
  InfoDeepSeek is a new benchmark for assessing agentic information seeking in real-world, dynamic web environments.<n>We propose a systematic methodology for constructing challenging queries satisfying the criteria of determinacy, difficulty, and diversity.<n>We develop the first evaluation framework tailored to dynamic agentic information seeking, including fine-grained metrics about the accuracy, utility, and compactness of information seeking outcomes.
  arXiv  Detail & Related papers  (2025-05-21T14:44:40Z)
• STaRK: Benchmarking LLM Retrieval on Textual and Relational Knowledge Bases [93.96463520716759]
  We develop STARK, a large-scale Semi-structure retrieval benchmark on Textual and Knowledge Bases. Our benchmark covers three domains: product search, academic paper search, and queries in precision medicine. We design a novel pipeline to synthesize realistic user queries that integrate diverse relational information and complex textual properties.
  arXiv  Detail & Related papers  (2024-04-19T22:54:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.