DRFLOW: A Deep Research Benchmark for Personalized Workflow Prediction

• URL: http://arxiv.org/abs/2606.18191v2
• Date: Wed, 17 Jun 2026 21:10:12 GMT
• Title: DRFLOW: A Deep Research Benchmark for Personalized Workflow Prediction
• Abstract summary: Many enterprise tasks require an agent to identify concrete which is a sequence of action-steps.<n>We introduce DRFLOW, a benchmark for evaluating personalized predictions from heterogeneous sources.<n>DRFLOW contains 100 tasks across five domains, with 1,246 reference workflow steps grounded in more than 3,900 sources.
• Score: 44.59825034567626
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep research (DR) systems are increasingly used for complex information-seeking tasks, but existing works mainly focus on generating reports and summaries. In contrast, many enterprise tasks instead require an agent to identify concrete workflows which is a sequence of action-steps. For example, rather than summarizing budgeting policies, an agent should be able to determine the steps needed to answer a question such as: "How do I request new headcount given a fixed budget?". Therefore, we introduce DRFLOW, a benchmark for evaluating personalized workflows predicted by agents from heterogeneous sources. Each task requires the agent to identify relevant evidence from scattered sources, then use that evidence to predict the correct action-step sequence for the user's task. DRFLOW contains 100 tasks across five domains, with 1,246 reference workflow steps grounded in more than 3,900 sources. We define seven diagnostic metrics covering factual grounding, step recovery, structural ordering, condition resolution, and personalization. We further present DRFLOW-Agent (DRFA), a workflow-oriented reference agent to predict personalized workflow. We show that although DRFA improves over strong baseline agents (upto 10.02% average F1 score), there is substantial room for improvement remains across these workflow metrics, indicating that predicting complete and correct personalized workflows remains a challenging frontier for deep research.

Related papers

• Beyond Generalist LLMs: Specialist Agentic Systems for Structured Code Workflow Execution [0.40631409309544836]
  Large Language Models (LLMs) have accelerated the adoption of software development agents, now widely available as Integrated Development Environment (IDE) extensions and standalone applications.<n>While these agents are typically general-purpose, it remains unclear whether specialist agents justify their additional development effort.<n>We investigate this question in the context of business process automation, focusing on the transformation of Business Process Model and Notation (BPMN) diagrams into executable agentic.
  arXiv  Detail & Related papers  (2026-07-16T01:06:38Z)
• AutoMedBench: Towards Medical AutoResearch with Agentic AI Models [31.335309699875264]
  AutoMedBench is a workflow-aware benchmark for autonomous medical-AI research.<n>It comprises long-horizon tasks with each run averaging 33 agent turns.
  arXiv  Detail & Related papers  (2026-06-01T09:22:55Z)
• From Research Question to Scientific Workflow: Leveraging Agentic AI for Science Automation [0.0]
  We implement and evaluate the architecture on the 1000 Genomes population genetics workflow running on Hyperflow WMS.<n>In an ablation study on 150 queries, Skills raise full-match intent accuracy from 44% to 83%; skill-driven deferred workflow generation reduces data transfer by 92%.
  arXiv  Detail & Related papers  (2026-04-23T17:52:52Z)
• Do We Always Need Query-Level Workflows? Rethinking Agentic Workflow Generation for Multi-Agent Systems [72.3575737073235]
  Multi-Agent Systems (MAS) solve complex tasks by coordinating multiple agents through.<n>Existing approaches generates either at task level or query level, but their relative costs and benefits remain unclear.<n>We show that query-level workflow generation is not always necessary, since a small set of top-K best task-level together already covers equivalent or even more queries.
  arXiv  Detail & Related papers  (2026-01-16T10:05:51Z)
• Beyond Rule-Based Workflows: An Information-Flow-Orchestrated Multi-Agents Paradigm via Agent-to-Agent Communication from CORAL [0.15199492741752027]
  We propose an Information-Flow-Orchestrated Multi-Agent Paradigm via Agent-to-Agent (A2A) Communication.<n>We evaluate our approach on the general-purpose benchmark GAIA, using the representative workflow-based MAS as the baseline.<n>Our method achieves 63.64% accuracy, outperforming OWL's 55.15% by 8.49 percentage points with comparable token consumption.
  arXiv  Detail & Related papers  (2026-01-14T21:35:51Z)
• AgentPRM: Process Reward Models for LLM Agents via Step-Wise Promise and Progress [71.02263260394261]
  Large language models (LLMs) still encounter challenges in multi-turn decision-making tasks.<n>We build process reward models (PRMs) to evaluate each decision and guide the agent's decision-making process.<n>AgentPRM captures both the interdependence between sequential decisions and their contribution to the final goal.
  arXiv  Detail & Related papers  (2025-11-11T14:57:54Z)
• Agentic Predictor: Performance Prediction for Agentic Workflows via Multi-View Encoding [56.565200973244146]
  Agentic Predictor is a lightweight predictor for efficient agentic workflow evaluation.<n>By learning to approximate task success rates, Agentic Predictor enables fast and accurate selection of optimal agentic workflow configurations.
  arXiv  Detail & Related papers  (2025-05-26T09:46:50Z)
• Turning Conversations into Workflows: A Framework to Extract and Evaluate Dialog Workflows for Service AI Agents [65.36060818857109]
  We present a novel framework for extracting and evaluating dialog from historical interactions.<n>Our extraction process consists of two key stages: (1) a retrieval step to select relevant conversations based on key procedural elements, and (2) a structured workflow generation process using a question-answer-based chain-of-thought (QA-CoT) prompting.
  arXiv  Detail & Related papers  (2025-02-24T16:55:15Z)
• Benchmarking Agentic Workflow Generation [80.74757493266057]
  We introduce WorfBench, a unified workflow generation benchmark with multi-faceted scenarios and intricate graph workflow structures.<n>We also present WorfEval, a systemic evaluation protocol utilizing subsequence and subgraph matching algorithms.<n>We observe that the generated can enhance downstream tasks, enabling them to achieve superior performance with less time during inference.
  arXiv  Detail & Related papers  (2024-10-10T12:41:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.