BatchDAG: LLM-Planned Execution Graphs for Scalable Ad-Hoc Analysis Over Enterprise Data

• URL: http://arxiv.org/abs/2607.18241v1
• Date: Fri, 17 Apr 2026 19:22:34 GMT
• Title: BatchDAG: LLM-Planned Execution Graphs for Scalable Ad-Hoc Analysis Over Enterprise Data
• Abstract summary: BatchDAG is a system in which an LLM generates a typed acyclic graph (DAG) of operations.<n> BatchDAG processes queries over 50,000+ meetings in under 60 seconds.<n>In production at Brevian, BatchDAG processes queries over 50,000+ meetings in under 60 seconds.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) excel at analyzing individual documents but break down on exhaustive, cross-entity analytical questions over enterprise-scale datasets due to context overflow, loss of per-entity attribution, and linear latency from sequential tool calls. We present BatchDAG, a system in which an LLM generates a typed directed acyclic graph (DAG) of operations -- SQL queries, semantic searches, in-memory transforms, parallel fan-outs, and single-shot analyses -- which a deterministic engine evaluates with topological-wave parallelism and structured JSON data flow. A key optimization, entity-aware batching, groups rows by logical entity before fan-out, reducing LLM calls by up to 47x. BatchDAG is not primarily an accuracy improvement over hand-optimized pipelines; rather, it is a general-purpose orchestration layer that replaces multiple hand-engineered workflows with a single system that generates the appropriate execution strategy from natural language. In controlled experiments on 12 transcript-heavy queries, BatchDAG (3.74/5) achieves quality comparable to an expert-designed pipeline (3.25/5) and significantly outperforms a ReAct agent (3.09/5, p<0.01), with superior provenance (77% transcript evidence rate vs. 46-60% for baselines). A controlled ablation shows structured JSON intermediates reduce hallucinations by 27% versus prose summaries (paired t-test, p=0.107, n=12). The planner achieves 98.8% valid-DAG rate across 300 planning calls. In production at Brevian.ai, BatchDAG processes queries over 50,000+ meetings in under 60 seconds, with measured per-query costs of $0.02-$0.24 at published GPT-5.1 pricing.

Related papers

• APEX: Automated Prompt Engineering eXpert with Dynamic Data Selection [60.504476571531]
  Large Language Models are highly sensitive to prompt formulation, necessitating automatic prompt optimization to unlock their full potential.<n>Current methods treat the development dataset as a static benchmark, wasting significant compute budget on uninformative data.<n>We introduce APEX (Automatic Prompt Engineering eXpert), a novel framework that optimize the data usage alongside the prompt search.
  arXiv  Detail & Related papers  (2026-06-09T21:22:06Z)
• BADGER: Bridging Agentic and Deterministic Evaluation for Generative Enterprise Reasoning [0.8681424810073922]
  BADGER is a unified framework integrating text-to-assessment with agentic behavior evaluation.<n> BADGER runs entirely within client's governed data environment.<n>It supports rapid prototyping of client-specific judges and metrics.
  arXiv  Detail & Related papers  (2026-06-01T11:42:37Z)
• Knowledge Graphs as the Missing Data Layer for LLM-Based Industrial Asset Operations [0.0]
  LLM-based agents for industrial asset operations show limited accuracy when reasoning over flat document stores.<n> AssetOpsBench establishes that GPT-4 agents achieve 65% on 139 industrial maintenance scenarios.<n>We treat a typed knowledge graph as a grounding substrate and route each question by how it is best answered.
  arXiv  Detail & Related papers  (2026-05-26T11:31:46Z)
• Models Can Model, But Can't Bind: Structured Grounding in Text-to-Optimization [54.749573452394664]
  We find that accuracy collapses as instance data grows, even when the formulation itself is simple.<n>We address this via a simple inference-time approach, BIND, which externalizes numeric data to structured files so the model binds data rather than from prompt prompt.<n>We validate our hypothesis by finetuning a model exclusively on binding and show that it outperforms end-to-end SFT and RL across three structurally distinct optimization categories.
  arXiv  Detail & Related papers  (2026-05-20T21:25:41Z)
• optimize_anything: A Universal API for Optimizing any Text Parameter [98.42497715725356]
  We show a single AI-based optimization system-supporting single-task search, multi-task search with cross-problem transfer, and generalization to unseen inputs.<n>We show for the first time that text optimization with LLM-based search is a general-purpose problem-solving paradigm.
  arXiv  Detail & Related papers  (2026-05-19T10:18:12Z)
• ComplexMCP: Evaluation of LLM Agents in Dynamic, Interdependent, and Large-Scale Tool Sandbox [61.862814740220806]
  $textbfComplexMCP$ is a benchmark designed to evaluate agents in rigorous conditions.<n>Built on the Model Context Protocol (MCP), $textbfComplexMCP$ provides over 300 meticulously tested tools.
  arXiv  Detail & Related papers  (2026-05-11T16:20:51Z)
• PlanCompiler: A Deterministic Compilation Architecture for Structured Multi-Step LLM Pipelines [0.0]
  We present PlanCompiler, a compilation architecture for structured pipelines that separates planning from execution through a typed node registry registry.<n>PlanCompiler produces a typed plan over a fixed registry of primitives, validates that plan against explicit structural and type constraints, and compiles only validated plans into executable Python.
  arXiv  Detail & Related papers  (2026-04-08T00:54:41Z)
• APEX-EM: Non-Parametric Online Learning for Autonomous Agents via Structured Procedural-Episodic Experience Replay [7.370176470430802]
  LLM-based autonomous agents lack persistent procedural memory.<n>We present APEX-EM, a non-parametric online learning framework that accumulates, retrieves, and reuses structured procedural plans.
  arXiv  Detail & Related papers  (2026-03-31T00:24:56Z)
• The Limits of Long-Context Reasoning in Automated Bug Fixing [4.853967615615349]
  Large language models (LLMs) can directly reason over entire contexts.<n>Recent advances in LLMs have enabled strong performance on software engineering benchmarks.<n>We systematically evaluate whether current LLMs can reliably perform long-context code and patch generation.
  arXiv  Detail & Related papers  (2026-02-17T22:51:40Z)
• RAG-Driven Data Quality Governance for Enterprise ERP Systems [0.0]
  We present an end-to-end pipeline combining automated data cleaning with LLM-driven query generation.<n>The system is deployed on a production system managing 240,000 employee records over six months.<n>This modular architecture provides a reproducible framework for AI-native enterprise data governance.
  arXiv  Detail & Related papers  (2025-11-18T12:08:44Z)
• Towards Global Retrieval Augmented Generation: A Benchmark for Corpus-Level Reasoning [50.27838512822097]
  We introduce GlobalQA, the first benchmark specifically designed to evaluate global RAG capabilities.<n>We propose GlobalRAG, a multi-tool collaborative framework that preserves structural coherence through chunk-level retrieval.<n>On the Qwen2.5-14B model, GlobalRAG achieves 6.63 F1 compared to the strongest baseline's 1.51 F1.
  arXiv  Detail & Related papers  (2025-10-30T07:29:14Z)
• SOPBench: Evaluating Language Agents at Following Standard Operating Procedures and Constraints [59.645885492637845]
  SOPBench is an evaluation pipeline that transforms each service-specific SOP code program into a directed graph of executable functions.<n>Our approach transforms each service-specific SOP code program into a directed graph of executable functions and requires agents to call these functions based on natural language SOP descriptions.<n>We evaluate 18 leading models, and results show the task is challenging even for top-tier models.
  arXiv  Detail & Related papers  (2025-03-11T17:53:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.