CODA-BENCH: Can Code Agents Handle Data-Intensive Tasks?

• URL: http://arxiv.org/abs/2606.15300v1
• Date: Sat, 13 Jun 2026 13:46:47 GMT
• Title: CODA-BENCH: Can Code Agents Handle Data-Intensive Tasks?
• Abstract summary: CODA-BENCH is the first benchmark to jointly evaluate code and data intelligence in a data-intensive environment.<n>We construct a data-intensive Linux sandbox based on the Kaggle ecosystem.<n> CODA-BENCH comprises 1,009 tasks spanning 31 communities, with each task environment containing an average of 980 files.
• Score: 37.616773601723416
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Advanced agents are increasingly demonstrating the potential to operate as autonomous engineers, creating a growing demand for evaluation benchmarks that capture the complexity of real-world development. Such environments typically involve both complex code and large-scale data (i.e., file system). However, existing benchmarks usually evaluate code-centric or data-centric capabilities in isolation, leaving a clear gap with real development scenarios. In this paper, we bridge this gap by introducing CODA-BENCH, the first benchmark to jointly evaluate code and data intelligence in a data-intensive environment. We construct a data-intensive Linux sandbox based on the Kaggle ecosystem (containing hundreds of datasets), where agents must actively explore complex file hierarchies to identify relevant resources and generate code for data-driven analytical tasks. CODA-BENCH comprises 1,009 tasks spanning 31 communities, with each task environment containing an average of 980 files, simulating realistic data scale and noise. Evaluations of advanced agents reveal that even top-performing systems struggle to effectively integrate data discovery with code execution, achieving a success rate of only 61.1%. These results highlight a substantial gap in current agentic capabilities for data-intensive tasks and point to promising directions for future research.

Related papers

• AgenticDataBench: A Comprehensive Benchmark for Data Agents [26.292243502468335]
  Large language model (MLL) data agents have emerged as a promising solution to automate data science.<n>We propose AgenticData, a comprehensive benchmark featuring realistic tasks spanning diverse domains with fine-grained granularity.<n>This enables evaluations to capture the diversity and complexity of data science and the detailed performance of agents.
  arXiv  Detail & Related papers  (2026-07-02T03:18:59Z)
• DAComp: Benchmarking Data Agents across the Full Data Intelligence Lifecycle [41.576417987200074]
  Real-world enterprise data intelligence encompass data engineering that turns raw sources into analytical-ready tables and data analysis that convert those tables into decision-oriented insights.<n>We introduce DAComp, a benchmark of 210 tasks that mirrors these complex capabilities.
  arXiv  Detail & Related papers  (2025-12-03T23:21:28Z)
• CoDA: Agentic Systems for Collaborative Data Visualization [57.270599188947294]
  Deep research has revolutionized data analysis, yet data scientists still devote substantial time to manually crafting visualizations.<n>Existing approaches, including simple single- or multi-agent systems, often oversimplify the task.<n>We introduce CoDA, a multi-agent system that employs specialized LLM agents for metadata analysis, task planning, code generation, and self-reflection.
  arXiv  Detail & Related papers  (2025-10-03T17:30:16Z)
• Scaling Generalist Data-Analytic Agents [95.05161133349242]
  DataMind is a scalable data synthesis and agent training recipe designed to build generalist data-analytic agents.<n>DataMind tackles three key challenges in building open-source data-analytic agents.
  arXiv  Detail & Related papers  (2025-09-29T17:23:08Z)
• DatasetResearch: Benchmarking Agent Systems for Demand-Driven Dataset Discovery [26.388978716803464]
  Can AI agents transcend conventional search to systematically discover any dataset that meets specific user requirements?<n>Our benchmark and comprehensive analysis provide the foundation for the next generation of self-improving AI systems.
  arXiv  Detail & Related papers  (2025-08-09T12:15:08Z)
• Benchmarking Deep Search over Heterogeneous Enterprise Data [73.55304268238474]
  We present a new benchmark for evaluating a form of retrieval-augmented generation (RAG)<n>RAG requires source-aware, multi-hop reasoning over diverse, sparsed, but related sources.<n>We build it using a synthetic data pipeline that simulates business across product planning, development, and support stages.
  arXiv  Detail & Related papers  (2025-06-29T08:34:59Z)
• Towards an Introspective Dynamic Model of Globally Distributed Computing Infrastructures [27.473508984130728]
  Large-scale scientific collaborations generate petabytes of data, with volumes soon expected to reach exabytes.<n>To manage these computational and storage demands, centralized workflow and data management systems are implemented.<n>A significant obstacle in adopting more effective or AI-driven solutions is the absence of a quick and reliable introspective dynamic model.
  arXiv  Detail & Related papers  (2025-06-24T12:42:36Z)
• Codev-Bench: How Do LLMs Understand Developer-Centric Code Completion? [60.84912551069379]
  We present the Code-Development Benchmark (Codev-Bench), a fine-grained, real-world, repository-level, and developer-centric evaluation framework. Codev-Agent is an agent-based system that automates repository crawling, constructs execution environments, extracts dynamic calling chains from existing unit tests, and generates new test samples to avoid data leakage.
  arXiv  Detail & Related papers  (2024-10-02T09:11:10Z)
• DiscoveryBench: Towards Data-Driven Discovery with Large Language Models [50.36636396660163]
  We present DiscoveryBench, the first comprehensive benchmark that formalizes the multi-step process of data-driven discovery. Our benchmark contains 264 tasks collected across 6 diverse domains, such as sociology and engineering. Our benchmark, thus, illustrates the challenges in autonomous data-driven discovery and serves as a valuable resource for the community to make progress.
  arXiv  Detail & Related papers  (2024-07-01T18:58:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.