Deep Research is the New Analytics System: Towards Building the Runtime for AI-Driven Analytics

• URL: http://arxiv.org/abs/2509.02751v1
• Date: Tue, 02 Sep 2025 18:59:15 GMT
• Title: Deep Research is the New Analytics System: Towards Building the Runtime for AI-Driven Analytics
• Abstract summary: We build a prototype which enables Deep Research agents to write and execute optimized semantic operator programs.<n>Compared to a standard open Deep Research agent, our prototype achieves up to 1.95x better F1-score.<n>Even if we give the agent access to semantic operators as tools, our prototype still achieves cost and runtime savings of up to 76.8% and 72.7%.
• Score: 6.298765290689598
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With advances in large language models (LLMs), researchers are creating new systems that can perform AI-driven analytics over large unstructured datasets. Recent work has explored executing such analytics queries using semantic operators -- a declarative set of AI-powered data transformations with natural language specifications. However, even when optimized, these operators can be expensive to execute on millions of records and their iterator execution semantics make them ill-suited for interactive data analytics tasks. In another line of work, Deep Research systems have demonstrated an ability to answer natural language question(s) over large datasets. These systems use one or more LLM agent(s) to plan their execution, process the dataset(s), and iteratively refine their answer. However, these systems do not explicitly optimize their query plans which can lead to poor plan execution. In order for AI-driven analytics to excel, we need a runtime which combines the optimized execution of semantic operators with the flexibility and more dynamic execution of Deep Research systems. As a first step towards this vision, we build a prototype which enables Deep Research agents to write and execute optimized semantic operator programs. We evaluate our prototype and demonstrate that it can outperform a handcrafted semantic operator program and open Deep Research systems on two basic queries. Compared to a standard open Deep Research agent, our prototype achieves up to 1.95x better F1-score. Furthermore, even if we give the agent access to semantic operators as tools, our prototype still achieves cost and runtime savings of up to 76.8% and 72.7% thanks to its optimized execution.

Related papers

• The Optimizer Is the Agent: Reasoning-Driven Search across Prompts, Programs, and ML Workflows [72.9051182568928]
  We present ReASearch, a unified framework for reasoning-driven optimization.<n>With a shared agent loop and domain-specific tools, ReASearch instantiates the exact same scaffold to optimize prompts, programs, and ML.<n>We observe that complex search behaviors emerge naturally from the agent's reasoning process.
  arXiv  Detail & Related papers  (2026-08-07T02:15:39Z)
• HepScript: A Dual-Use DSL for Human-AI Collaborative Data Analysis Workflows in High-Energy Physics [17.146017163668557]
  HepScript serves as a shared formal interface, abstracting analysis logic into a constrained syntax.<n>This abstraction reduces the required human-written code by 93%.<n>Our work demonstrates a scalable pathway toward human-AI collaborative systems.
  arXiv  Detail & Related papers  (2026-05-02T12:42:34Z)
• Let the Barbarians In: How AI Can Accelerate Systems Performance Research [80.43506848683633]
  We term this iterative cycle of generation, evaluation, and refinement AI-Driven Research for Systems.<n>We demonstrate that ADRS-generated solutions can match or even outperform human state-of-the-art designs.
  arXiv  Detail & Related papers  (2025-12-16T18:51:23Z)
• LoCoBench-Agent: An Interactive Benchmark for LLM Agents in Long-Context Software Engineering [90.84806758077536]
  We introduce textbfLoCoBench-Agent, a comprehensive evaluation framework specifically designed to assess large language models (LLMs) agents in realistic, long-context software engineering.<n>Our framework extends LoCoBench's 8,000 scenarios into interactive agent environments, enabling systematic evaluation of multi-turn conversations.<n>Our framework provides agents with 8 specialized tools (file operations, search, code analysis) and evaluates them across context lengths ranging from 10K to 1M tokens.
  arXiv  Detail & Related papers  (2025-11-17T23:57:24Z)
• Enterprise Deep Research: Steerable Multi-Agent Deep Research for Enterprise Analytics [75.4712507893024]
  Enterprise Deep Research (EDR) is a multi-agent system that integrates a Master Planning Agent for adaptive query decomposition.<n>Four specialized search agents (General, Academic, GitHub, LinkedIn) and a visualization agent for data-driven insights are also included.<n>EDR reflects research direction with optional human-in-the-loop steering guidance.
  arXiv  Detail & Related papers  (2025-10-20T17:55:11Z)
• DeepAnalyze: Agentic Large Language Models for Autonomous Data Science [35.69385623867138]
  We introduce DeepAnalyze-8B, the first agentic designed for autonomous data science.<n>We propose a curriculum-based agentic training paradigm that emulates the learning trajectory of human data scientists.<n>We also introduce a data-grounded trajectory synthesis framework that constructs high-quality training data.
  arXiv  Detail & Related papers  (2025-10-19T15:13:42Z)
• Spec-Driven AI for Science: The ARIA Framework for Automated and Reproducible Data Analysis [23.28226188948918]
  ARIA is a spec-driven, human-in-the-loop framework for automated and interpretable data analysis.<n>ARIA integrates six layers, namely Command, Context, Code, Data, Orchestration, and AI Module.<n>ARIA establishes a new paradigm for transparent, collaborative, and reproducible scientific discovery.
  arXiv  Detail & Related papers  (2025-10-13T08:32:43Z)
• AgenticData: An Agentic Data Analytics System for Heterogeneous Data [12.67277567222908]
  AgenticData is an agentic data analytics system that allows users to pose natural language (NL) questions while autonomously analyzing data sources across multiple domains.<n>We propose a multi-agent collaboration strategy by utilizing a data profiling agent for discovering relevant data, a semantic cross-validation agent for iterative optimization based on feedback, and a smart memory agent for maintaining short-term context.
  arXiv  Detail & Related papers  (2025-08-07T03:33:59Z)
• Deep Research Agents: A Systematic Examination And Roadmap [109.53237992384872]
  Deep Research (DR) agents are designed to tackle complex, multi-turn informational research tasks.<n>In this paper, we conduct a detailed analysis of the foundational technologies and architectural components that constitute DR agents.
  arXiv  Detail & Related papers  (2025-06-22T16:52:48Z)
• ResearchCodeAgent: An LLM Multi-Agent System for Automated Codification of Research Methodologies [16.90884865239373]
  We introduce ResearchCodeAgent, a novel multi-agent system to automate the codification of research methodologies.<n>The system bridges the gap between high-level research concepts and their practical implementation.<n>ResearchCodeAgent represents a significant step towards the research implementation process, potentially accelerating the pace of machine learning research.
  arXiv  Detail & Related papers  (2025-04-28T07:18:45Z)
• Pointwise Mutual Information as a Performance Gauge for Retrieval-Augmented Generation [78.28197013467157]
  We show that the pointwise mutual information between a context and a question is an effective gauge for language model performance.<n>We propose two methods that use the pointwise mutual information between a document and a question as a gauge for selecting and constructing prompts that lead to better performance.
  arXiv  Detail & Related papers  (2024-11-12T13:14:09Z)
• Data Analysis in the Era of Generative AI [56.44807642944589]
  This paper explores the potential of AI-powered tools to reshape data analysis, focusing on design considerations and challenges. We explore how the emergence of large language and multimodal models offers new opportunities to enhance various stages of data analysis workflow. We then examine human-centered design principles that facilitate intuitive interactions, build user trust, and streamline the AI-assisted analysis workflow across multiple apps.
  arXiv  Detail & Related papers  (2024-09-27T06:31:03Z)
• Semantic Operators: A Declarative Model for Rich, AI-based Data Processing [27.294714926492187]
  We introduce semantic operators, the first formalism for declarative and general-purpose AI-based transformations.<n>We propose several novel optimizations to accelerate semantic filtering, joining, group-by and top-k operations by up to $1,000times$.<n>We show that the semantic operator model is expressive, capturing state-of-the-art AI pipelines in a few operator calls.
  arXiv  Detail & Related papers  (2024-07-16T06:19:14Z)
• DACO: Towards Application-Driven and Comprehensive Data Analysis via Code Generation [83.30006900263744]
  Data analysis is a crucial analytical process to generate in-depth studies and conclusive insights. We propose to automatically generate high-quality answer annotations leveraging the code-generation capabilities of LLMs. Our DACO-RL algorithm is evaluated by human annotators to produce more helpful answers than SFT model in 57.72% cases.
  arXiv  Detail & Related papers  (2024-03-04T22:47:58Z)
• Can Large Language Models Serve as Data Analysts? A Multi-Agent Assisted Approach for Qualitative Data Analysis [6.592797748561459]
  Large Language Models (LLMs) have enabled collaborative human-bot interactions in Software Engineering (SE) We introduce a new dimension of scalability and accuracy in qualitative research, potentially transforming data interpretation methodologies in SE.
  arXiv  Detail & Related papers  (2024-02-02T13:10:46Z)
• MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation [96.71370747681078]
  We introduce MLAgentBench, a suite of 13 tasks ranging from improving model performance on CIFAR-10 to recent research problems like BabyLM. For each task, an agent can perform actions like reading/writing files, executing code, and inspecting outputs. We benchmark agents based on Claude v1.0, Claude v2.1, Claude v3 Opus, GPT-4, GPT-4-turbo, Gemini-Pro, and Mixtral and find that a Claude v3 Opus agent is the best in terms of success rate.
  arXiv  Detail & Related papers  (2023-10-05T04:06:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.