SAGE: Steerable Agentic Data Generation for Deep Search with Execution Feedback

• URL: http://arxiv.org/abs/2601.18202v1
• Date: Mon, 26 Jan 2026 06:37:56 GMT
• Title: SAGE: Steerable Agentic Data Generation for Deep Search with Execution Feedback
• Authors: Fangyuan Xu, Rujun Han, Yanfei Chen, Zifeng Wang, I-Hung Hsu, Jun Yan, Vishy Tirumalashetty, Eunsol Choi, Tomas Pfister, Chen-Yu Lee,
• Abstract summary: We propose an agentic pipeline that automatically generates high quality, difficulty-controlled deep search question-answer pairs.<n>Our pipeline, SAGE, consists of a data generator which proposes QA pairs and a search agent which attempts to solve the generated question.<n>Our intrinsic evaluation shows SAGE generates questions that require diverse reasoning strategies, while significantly increases the correctness and difficulty of the generated data.
• Score: 68.60326181052658
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep search agents, which aim to answer complex questions requiring reasoning across multiple documents, can significantly speed up the information-seeking process. Collecting human annotations for this application is prohibitively expensive due to long and complex exploration trajectories. We propose an agentic pipeline that automatically generates high quality, difficulty-controlled deep search question-answer pairs for a given corpus and a target difficulty level. Our pipeline, SAGE, consists of a data generator which proposes QA pairs and a search agent which attempts to solve the generated question and provide execution feedback for the data generator. The two components interact over multiple rounds to iteratively refine the question-answer pairs until they satisfy the target difficulty level. Our intrinsic evaluation shows SAGE generates questions that require diverse reasoning strategies, while significantly increases the correctness and difficulty of the generated data. Our extrinsic evaluation demonstrates up to 23% relative performance gain on popular deep search benchmarks by training deep search agents with our synthetic data. Additional experiments show that agents trained on our data can adapt from fixed-corpus retrieval to Google Search at inference time, without further training.

Related papers

• AgentIR: Reasoning-Aware Retrieval for Deep Research Agents [76.29382561831105]
  Deep Research agents generate explicit natural language reasoning before each search call.<n> Reasoning-Aware Retrieval embeds the agent's reasoning trace alongside its query.<n>DR- Synth generates Deep Research retriever training data from standard QA datasets.<n>AgentIR-4B achieves 68% accuracy with the open-weight agent Tongyi-DeepResearch.
  arXiv  Detail & Related papers  (2026-03-04T18:47:26Z)
• A Picture of Agentic Search [32.93760518790612]
  Information Retrieval operates under assumptions that no longer hold in practice.<n>Without adaptation, retrieval models risk satisfying neither humans, nor the emerging user segment of agents.<n>We develop a methodology for collecting all the data produced and consumed by agentic retrieval-augmented systems.
  arXiv  Detail & Related papers  (2026-02-19T16:32:34Z)
• BMGQ: A Bottom-up Method for Generating Complex Multi-hop Reasoning Questions from Semi-structured Data [8.52473384574856]
  We present an automated framework for generating high-difficulty, training-ready multi-hop questions from semi-structured knowledge sources.<n>The system grows diverse, logically labeled evidence clusters through Natural Language Inference (NLI)-based relation typing and diversity-aware expansion.
  arXiv  Detail & Related papers  (2025-10-28T07:43:15Z)
• Query Decomposition for RAG: Balancing Exploration-Exploitation [83.79639293409802]
  RAG systems address complex user requests by decomposing them into subqueries, retrieving potentially relevant documents for each, and then aggregating them to generate an answer.<n>We formulate query decomposition and document retrieval in an exploitation-exploration setting, where retrieving one document at a time builds a belief about the utility of a given sub-queries.<n>Our main finding is that estimating document relevance using rank information and human judgments yields a 35% gain in document-level precision, 15% increase in alpha-nDCG, and better performance on the downstream task of long-form generation.
  arXiv  Detail & Related papers  (2025-10-21T13:37:11Z)
• Synthesizing Agentic Data for Web Agents with Progressive Difficulty Enhancement Mechanisms [81.90219895125178]
  Web-based 'deep research' agents aim to solve complex question - answering tasks through long-horizon interactions with online tools.<n>These tasks remain challenging, as the underlying language models are often not optimized for long-horizon reasoning.<n>We introduce a two-pronged data synthesis pipeline that generates question - answer pairs by progressively increasing complexity.
  arXiv  Detail & Related papers  (2025-10-15T06:34:46Z)
• Beyond the limitation of a single query: Train your LLM for query expansion with Reinforcement Learning [23.104182075898297]
  Reasoning-augmented search agents, such as Search-R1, are trained to reason, search, and generate the final answer iteratively.<n>We train an LLM-based search agent with the native capability of query expansion through reinforcement learning.<n>With the assistance of the squeezer model, we discover that even a small-scale 3B LLM can demonstrate a strong capability of query expansion.
  arXiv  Detail & Related papers  (2025-10-11T04:23:30Z)
• Open Data Synthesis For Deep Research [17.22470203913576]
  We formalize Deep Research tasks with verifiable answers as Hierarchical Constraint Satisfaction Problems.<n>Existing benchmarks (e.g., Natural Questions, HotpotQA) fail to capture this complexity.<n>We introduce InfoSeek, a scalable scalable complex Deep Research tasks.
  arXiv  Detail & Related papers  (2025-08-30T06:02:56Z)
• Benchmarking Deep Search over Heterogeneous Enterprise Data [73.55304268238474]
  We present a new benchmark for evaluating a form of retrieval-augmented generation (RAG)<n>RAG requires source-aware, multi-hop reasoning over diverse, sparsed, but related sources.<n>We build it using a synthetic data pipeline that simulates business across product planning, development, and support stages.
  arXiv  Detail & Related papers  (2025-06-29T08:34:59Z)
• Benchmarking Multimodal Retrieval Augmented Generation with Dynamic VQA Dataset and Self-adaptive Planning Agent [92.5712549836791]
  Multimodal Retrieval Augmented Generation (mRAG) plays an important role in mitigating the "hallucination" issue inherent in multimodal large language models (MLLMs)<n>We propose the first self-adaptive planning agent for multimodal retrieval, OmniSearch.
  arXiv  Detail & Related papers  (2024-11-05T09:27:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.