"Skill issues'': data-centric optimization of lakehouse agents

• URL: http://arxiv.org/abs/2606.01185v1
• Date: Sun, 31 May 2026 11:58:04 GMT
• Title: "Skill issues'': data-centric optimization of lakehouse agents
• Abstract summary: We show how to optimize artifacts for agents operating on a branching lakehouse, Bauplan.<n>We present a data-centric optimization pipeline that generates task-verifier pairs, executes candidate skills in isolated sandboxes, and scores trajectories using both trace-level signals and programmatic checks over lakehouse state.
• Score: 2.8558664643005525
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Coding agents are becoming users of data infrastructure, but their success depends not only on model quality: it also depends on the skills and environment files that teach agents how to use a system. We study how to optimize these artifacts for agents operating on a branching lakehouse, Bauplan. In our setting, headless APIs and Git-like data primitives expose data workflows through code, branches, commits, and merges. Our central observation is that a branching lakehouse turns data-agent evaluation from an output-matching problem into a state-verification problem: agent-generated pipeline code induces concrete, inspectable lakehouse changes. We present a data-centric optimization pipeline that generates task-verifier pairs, executes candidate skills in isolated sandboxes, and scores trajectories using both trace-level signals and programmatic checks over lakehouse state. In a preliminary evaluation on 25 tasks, optimized skills improve accuracy by 31.9%. These results suggest that write-path data workflows provide a useful substrate for optimizing agent skills beyond read-only tasks.

Related papers

• FlowScout: From Execution Feedback to Reliable Tool-Using Agent Workflows [6.767792769288717]
  FlowScout is an execution-guided framework for generating tool-integrated agentic from historical task-solving records.<n>It first mines a common tool coordination from historical records to construct an initial workflow.<n>It then refines the workflow topology through Monte Carlo tree search guided by execution feedback.
  arXiv  Detail & Related papers  (2026-08-10T08:41:59Z)
• DataClaw0: Agentic Tailoring Multimodal Data from Raw Streams [48.434767119224375]
  We propose a paradigm shift towards Agentic Data Tailoring.<n>To overcome the data scarcity bottleneck in training such high-order capabilities, we design a two-stage pipeline.<n>$textDataClaw_0$-9B model synergizes Supervised Fine-Tuning with Group Relative Policy Optimization.
  arXiv  Detail & Related papers  (2026-06-19T11:31:43Z)
• SANA: What Matters for QA Agents over Massive Data Lakes? [21.461240255601684]
  We present SANA, a framework that transforms EQA tasks into runtime profiles containing gold source sequence, sanitized subquestions, and execution records.<n>We evaluate two recent EQA benchmarks, LakeQA and KramaBench, and evaluate lightweight and mid-sized agents under fixed prompts, budgets, data lakes, and runtimes.
  arXiv  Detail & Related papers  (2026-06-11T20:51:44Z)
• DataMaster: Data-Centric Autonomous AI Research [51.95819363392062]
  We study task-conditioned autonomous data engineering, where an agent improves a fixed learning algorithm by optimizing only the data side.<n>We propose DataMaster, a data-agent framework that integrates tree-structured search, shared candidate data, and cumulative memory.<n>We evaluate DataMaster on two types of benchmarks, MLE-Bench Lite and PostTrainBench.
  arXiv  Detail & Related papers  (2026-05-11T17:46:24Z)
• From Research Question to Scientific Workflow: Leveraging Agentic AI for Science Automation [0.0]
  We implement and evaluate the architecture on the 1000 Genomes population genetics workflow running on Hyperflow WMS.<n>In an ablation study on 150 queries, Skills raise full-match intent accuracy from 44% to 83%; skill-driven deferred workflow generation reduces data transfer by 92%.
  arXiv  Detail & Related papers  (2026-04-23T17:52:52Z)
• AgentOpt v0.1 Technical Report: Client-Side Optimization for LLM-Based Agent [18.58248061903799]
  We introduce AgentOpt, the first framework-agnostic Python package for client-side AI optimization.<n>We first study model selection, a high-impact optimization lever in multi-step agent pipelines.<n>To efficiently explore the exponentially growing combination space, AgentOpt implements ten search algorithms.
  arXiv  Detail & Related papers  (2026-04-07T17:13:47Z)
• SWE-QA-Pro: A Representative Benchmark and Scalable Training Recipe for Repository-Level Code Understanding [41.98672557723593]
  We introduce SWEQA-Pro, a benchmark constructed from diverse, long-tail repositories with executable environments.<n>We also propose a scalable synthetic data pipeline that powers a two-stage training recipe: Supervised Fine-Tuning (SFT) followed by Reinforcement Learning from AI Feedback (RLAIF)<n> Empirically, a Qwen3-8B model trained with our recipe surpasses GPT-4o by 2.3 points on SWE-QA-Pro and substantially narrows the gap to state-of-the-art models.
  arXiv  Detail & Related papers  (2026-03-17T05:12:48Z)
• Organizing, Orchestrating, and Benchmarking Agent Skills at Ecosystem Scale [28.43462779191672]
  AgentSkillOS is a principled framework for skill selection, orchestration, and ecosystem-level management.<n>AgentSkillOS comprises two stages: (i) Manage Skills, which organizes skills into a capability tree.<n> (ii) Solve Tasks, which retrieves, orchestrates, and executes multiple skills through DAG-based pipelines.
  arXiv  Detail & Related papers  (2026-03-02T18:46:47Z)
• ISO-Bench: Can Coding Agents Optimize Real-World Inference Workloads? [0.8749675983608171]
  We introduce ISO-Bench, a benchmark for coding agents to test their capabilities on real-world inference tasks.<n>We curated 54 tasks from merged pull requests with measurable performance improvements.
  arXiv  Detail & Related papers  (2026-02-23T08:37:53Z)
• ET-Agent: Incentivizing Effective Tool-Integrated Reasoning Agent via Behavior Calibration [68.89572566071575]
  ETAgent is a training framework for calibrating agent's tool-use behavior.<n>It is designed to progressively calibrate erroneous behavioral patterns to optimal behaviors.
  arXiv  Detail & Related papers  (2026-01-11T11:05:26Z)
• MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation [96.71370747681078]
  We introduce MLAgentBench, a suite of 13 tasks ranging from improving model performance on CIFAR-10 to recent research problems like BabyLM. For each task, an agent can perform actions like reading/writing files, executing code, and inspecting outputs. We benchmark agents based on Claude v1.0, Claude v2.1, Claude v3 Opus, GPT-4, GPT-4-turbo, Gemini-Pro, and Mixtral and find that a Claude v3 Opus agent is the best in terms of success rate.
  arXiv  Detail & Related papers  (2023-10-05T04:06:12Z)
• Nemo: Guiding and Contextualizing Weak Supervision for Interactive Data Programming [77.38174112525168]
  We present Nemo, an end-to-end interactive Supervision system that improves overall productivity of WS learning pipeline by an average 20% (and up to 47% in one task) compared to the prevailing WS supervision approach.
  arXiv  Detail & Related papers  (2022-03-02T19:57:32Z)
• Retrieval-Augmented Reinforcement Learning [63.32076191982944]
  We train a network to map a dataset of past experiences to optimal behavior. The retrieval process is trained to retrieve information from the dataset that may be useful in the current context. We show that retrieval-augmented R2D2 learns significantly faster than the baseline R2D2 agent and achieves higher scores.
  arXiv  Detail & Related papers  (2022-02-17T02:44:05Z)
• How Useful is Self-Supervised Pretraining for Visual Tasks? [133.1984299177874]
  We evaluate various self-supervised algorithms across a comprehensive array of synthetic datasets and downstream tasks. Our experiments offer insights into how the utility of self-supervision changes as the number of available labels grows.
  arXiv  Detail & Related papers  (2020-03-31T16:03:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.