Arming Data Agents with Tribal Knowledge

• URL: http://arxiv.org/abs/2602.13521v2
• Date: Tue, 17 Feb 2026 21:28:23 GMT
• Title: Arming Data Agents with Tribal Knowledge
• Abstract summary: NL2 agents make mistakes when faced with large-scale real-world databases.<n>We propose Tk-Boost, a framework for augmenting relational knowledge with tribal knowledge.<n>Tk-Boost identifies the agent's misconceptions by analyzing its mistakes on the database, and generates tribal knowledge to address them.
• Score: 19.74439813965971
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Natural language to SQL (NL2SQL) translation enables non-expert users to query relational databases through natural language. Recently, NL2SQL agents, powered by the reasoning capabilities of Large Language Models (LLMs), have significantly advanced NL2SQL translation. Nonetheless, NL2SQL agents still make mistakes when faced with large-scale real-world databases because they lack knowledge of how to correctly leverage the underlying data (e.g., knowledge about the intent of each column) and form misconceptions about the data when querying it, leading to errors. Prior work has studied generating facts about the database to provide more context to NL2SQL agents, but such approaches simply restate database contents without addressing the agent's misconceptions. In this paper, we propose Tk-Boost, a bolt-on framework for augmenting any NL2SQL agent with tribal knowledge: knowledge that corrects the agent's misconceptions in querying the database accumulated through experience using the database. To accumulate experience, Tk-Boost first asks the NL2SQL agent to answer a few queries on the database, identifies the agent's misconceptions by analyzing its mistakes on the database, and generates tribal knowledge to address them. To enable accurate retrieval, Tk-Boost indexes this knowledge with applicability conditions that specify the query features for which the knowledge is useful. When answering new queries, Tk-Boost uses this knowledge to provide feedback to the NL2SQL agent, resolving the agent's misconceptions during SQL generation, and thus improving the agent's accuracy. Extensive experiments across the BIRD and Spider 2.0 benchmarks with various NL2SQL agents shows Tk-Boost improves NL2SQL agents accuracy by up to 16.9% on Spider 2.0 and 13.7% on BIRD

Related papers

• Spider 2.0-AIFunc: Extending Real-World Text-to-SQL to AI-Native SQL Workflows [54.49251107006261]
  We introduce Spider 2.0-AIFunc, a benchmark of 465 verified instances across 125 real-world databases covering six types of AI functions on the Snowflake platform.<n>We find that the strongest proprietary models reach 67-70% execution accuracy while the best open-source model achieves 58.1%.
  arXiv  Detail & Related papers  (2026-07-07T12:54:08Z)
• AgentNLQ: A General-Purpose Agent for Natural Language to SQL [2.224371215696068]
  This study presents a new multi-agent method for NL2 that achieves 78.1% semantic accuracy on the BIg Bench for LaRge-scale Database benchmark.
  arXiv  Detail & Related papers  (2026-05-18T18:33:15Z)
• Monte Carlo Tree Search with Reasoning Path Refinement for Small Language Models in Conversational Text-to-NoSQL [20.156191782890797]
  We introduce the Conversational Text-to-No task, which generates queries given a natural language question, a database, and a dialogue history.<n>We propose Stage-MCTS, a framework that endows small language models with query-specific reasoning capabilities.<n>Our approach outperforms state-of-the-art large reasoning models, improving execution value match accuracy by up to 7.93%.
  arXiv  Detail & Related papers  (2026-02-13T03:35:38Z)
• From Queries to Insights: Agentic LLM Pipelines for Spatio-Temporal Text-to-SQL [8.496933324334167]
  We present a naive text-to-Act baseline (Rellama-sqlcoder-8b) with orchestration by a Mistral-based Rellama-sqlcoder-8b.<n>We evaluate on 35 natural-language queries over the NYC and Tokyo check-in, covering spatial, temporal multi-dataset reasoning.<n>The agent achieves substantially higher accuracy than the dataset 91.4% vs. 28.6% and enhances usability through maps, and plots structured natural-language summaries.
  arXiv  Detail & Related papers  (2025-10-29T22:18:57Z)
• End-to-End Text-to-SQL with Dataset Selection: Leveraging LLMs for Adaptive Query Generation [6.5390580456423555]
  Traditional approaches model text-to- query as a direct translation task.<n>Recent advances in large language models (LLMs) have significantly improved translation accuracy.<n>We propose a three-stage end-to-end text-to-end framework to identify the user's intended database.
  arXiv  Detail & Related papers  (2025-08-08T15:16:36Z)
• RAISE: Reasoning Agent for Interactive SQL Exploration [47.77323087050061]
  We propose a novel framework that unifies schema linking, query generation, and iterative refinement within a single, end-to-end component.<n>Our method emulates how humans answer questions when working with unfamiliar databases.
  arXiv  Detail & Related papers  (2025-06-02T03:07:08Z)
• TailorSQL: An NL2SQL System Tailored to Your Query Workload [16.48291142955493]
  State-of-the-art NL2 techniques typically perform translation by retrieving database-specific information.<n>We introduce Tailor, a NL2 system that takes advantage of information in the past query workload.<n> Tailor achieves up to 2$times$ improvement in execution accuracy on standardized benchmarks.
  arXiv  Detail & Related papers  (2025-05-29T03:27:22Z)
• Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows [64.94146689665628]
  Spider 2.0 is an evaluation framework for real-world text-to-sql problems derived from enterprise-level database use cases.<n>The databases in Spider 2.0 are sourced from real data applications, often containing over 1,000 columns and stored in local or cloud database systems such as BigQuery and Snowflake.<n>We show that solving problems in Spider 2.0 frequently requires understanding and searching through database metadata, dialect documentation, and even project-levels.
  arXiv  Detail & Related papers  (2024-11-12T12:52:17Z)
• A Survey of Text-to-SQL in the Era of LLMs: Where are we, and where are we going? [32.84561352339466]
  We provide a review of Text-to- translation techniques powered by Large Language Models (LLMs)<n>We discuss the research challenges and open problems of Text-to- evaluation in the LLMs era.
  arXiv  Detail & Related papers  (2024-08-09T14:59:36Z)
• MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQL [47.120862170230566]
  Recent Text-to-yourself methods usually suffer from significant performance degradation on "huge" databases.<n>We introduce MAC, a novel Text-to-yourself LLM-based multi-agent collaborative framework.<n>In our framework, we leverage GPT-4 as the strong backbone for all agent tasks to determine the upper bound of our framework.<n>We then fine-tune an open-sourced instruction-followed model,sql-Llama, by leveraging Code 7B, to accomplish all tasks as GPT-4 does.
  arXiv  Detail & Related papers  (2023-12-18T14:40:20Z)
• Natural language to SQL in low-code platforms [0.0]
  We propose a pipeline allowing developers to write natural language (NL) queries. We collect, label, and validate data covering the queries most often performed by OutSystems users. We describe the entire pipeline, which comprises a feedback loop that allows us to quickly collect production data.
  arXiv  Detail & Related papers  (2023-08-29T11:59:02Z)
• SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL (extended) [53.95151604061761]
  This paper introduces the framework for enhancing Text-to- filtering using large language models (LLMs) With few-shot prompting, we explore the effectiveness of consistency decoding with execution-based error analyses. With instruction fine-tuning, we delve deep in understanding the critical paradigms that influence the performance of tuned LLMs.
  arXiv  Detail & Related papers  (2023-05-26T21:39:05Z)
• UNITE: A Unified Benchmark for Text-to-SQL Evaluation [72.72040379293718]
  We introduce a UNIfied benchmark for Text-to-domain systems. It is composed of publicly available text-to-domain datasets and 29K databases. Compared to the widely used Spider benchmark, we introduce a threefold increase in SQL patterns.
  arXiv  Detail & Related papers  (2023-05-25T17:19:52Z)
• Interactive Text-to-SQL Generation via Editable Step-by-Step Explanations [31.3376894001311]
  We introduce a new interaction mechanism that allows users to directly edit a step-by-step explanation of a query to fix errors. Our experiments on multiple datasets, as well as a user with 24 participants, demonstrate that our approach can achieve better than multiple SOTA approaches.
  arXiv  Detail & Related papers  (2023-05-12T10:45:29Z)
• Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs [89.68522473384522]
  We present Bird, a big benchmark for large-scale database grounded in text-to-efficient tasks. Our emphasis on database values highlights the new challenges of dirty database contents. Even the most effective text-to-efficient models, i.e. ChatGPT, achieves only 40.08% in execution accuracy.
  arXiv  Detail & Related papers  (2023-05-04T19:02:29Z)
• Weakly Supervised Text-to-SQL Parsing through Question Decomposition [53.22128541030441]
  We take advantage of the recently proposed question meaning representation called QDMR. Given questions, their QDMR structures (annotated by non-experts or automatically predicted) and the answers, we are able to automatically synthesizesql queries. Our results show that the weakly supervised models perform competitively with those trained on NL- benchmark data.
  arXiv  Detail & Related papers  (2021-12-12T20:02:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.