EntSQL: A Benchmark for Grounding Text-to-SQL in Long-Context Enterprise Knowledge

• URL: http://arxiv.org/abs/2606.03363v1
• Date: Tue, 02 Jun 2026 09:12:57 GMT
• Title: EntSQL: A Benchmark for Grounding Text-to-SQL in Long-Context Enterprise Knowledge
• Abstract summary: Existing benchmarks such as Spider, BIRD, and Spider2.0 evaluate schema generalization, large-scale databases, and realistic.<n>We introduce Ent, an enterprise-oriented Text-to-private benchmark for evaluating long-context grounding over proprietary business documents.<n>On English inputs, the best evaluated system reaches only 15.9% when long-form documents are provided.
• Score: 20.59995531311023
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Text-to-SQL enables natural language access to databases, and recent LLMs have substantially advanced its capabilities. Existing benchmarks such as Spider, BIRD, and Spider~2.0 evaluate schema generalization, large-scale databases, and realistic workflows, but largely overlook enterprise scenarios where SQL generation depends on private business knowledge, such as internal metrics, reporting conventions, and organizational rules. We introduce EntSQL, an enterprise-oriented Text-to-SQL benchmark for evaluating long-context grounding over proprietary business documents. EntSQL contains 1,066 aligned Chinese-English semantic examples across five business domains, with most examples requiring domain knowledge beyond the question and schema and involving complex SQL structures. On English inputs, the best evaluated system reaches only 15.9\% when long-form documents are provided, highlighting the difficulty of grounding SQL generation in enterprise knowledge.

Related papers

• BIRD-History: A Benchmark for History-Driven Text-to-SQL with Fine-Grained Knowledge Annotations [28.456885492014475]
  BIRD-History is a benchmark for evaluating text-to-sourced systems' ability to ground underspecified natural language questions.<n>We propose a plug-in retriever that extracts five types of external knowledge from historical SQL scripts, then retrieves and reranks relevant fragments for query generation.<n>Experiments demonstrate consistent improvements across four text-to-sourced systems.
  arXiv  Detail & Related papers  (2026-08-29T15:59:56Z)
• ProSPy: A Profiling-Driven SQL-Python Agentic Framework for Enterprise Text-to-SQL [51.98164069124653]
  ProSPy is a Profiling-driven framework for enterprise-scale Text-to-gnostic analysis.<n>ProSPy structures the reasoning process into four stages.<n>It first extracts fine-grained data evidence through automatic profiling.<n>It prunes large schemas into task-relevant contexts.
  arXiv  Detail & Related papers  (2026-06-04T08:13:05Z)
• SING-SQL: A Synthetic Data Generation Framework for In-Domain Text-to-SQL Translation [2.0799061948689306]
  SING-a is a fully automated two-stage framework for generating high-quality, high-coverage synthetic Text-to-data.<n>SING-LM is a family of compact language models fine-tuned on the synthetic data.
  arXiv  Detail & Related papers  (2025-09-30T02:14:49Z)
• Datrics Text2SQL: A Framework for Natural Language to SQL Query Generation [0.0]
  This paper introduces a Retrieval-Augmented Generation (RAG)-based framework designed to generate accuratesql queries by leveraging structured documentation, example-based learning, and domain-specific rules.<n>The paper details the architecture, training methodology, and retrieval logic, highlighting how the system bridges the gap between user intent and database structure without requiringsql expertise.
  arXiv  Detail & Related papers  (2025-04-03T21:09:59Z)
• Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows [64.94146689665628]
  Spider 2.0 is an evaluation framework for real-world text-to-sql problems derived from enterprise-level database use cases.<n>The databases in Spider 2.0 are sourced from real data applications, often containing over 1,000 columns and stored in local or cloud database systems such as BigQuery and Snowflake.<n>We show that solving problems in Spider 2.0 frequently requires understanding and searching through database metadata, dialect documentation, and even project-levels.
  arXiv  Detail & Related papers  (2024-11-12T12:52:17Z)
• SQLfuse: Enhancing Text-to-SQL Performance through Comprehensive LLM Synergy [24.919119901664843]
  This paper introduces a robust system integrating open-source Large Language Models (LLMs) with a suite of tools to enhance query accuracy and usability. demonstrated by its leading performance on the Spider Leaderboard and deployment by Ant Group.
  arXiv  Detail & Related papers  (2024-07-19T06:01:57Z)
• SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL (extended) [53.95151604061761]
  This paper introduces the framework for enhancing Text-to- filtering using large language models (LLMs) With few-shot prompting, we explore the effectiveness of consistency decoding with execution-based error analyses. With instruction fine-tuning, we delve deep in understanding the critical paradigms that influence the performance of tuned LLMs.
  arXiv  Detail & Related papers  (2023-05-26T21:39:05Z)
• UNITE: A Unified Benchmark for Text-to-SQL Evaluation [72.72040379293718]
  We introduce a UNIfied benchmark for Text-to-domain systems. It is composed of publicly available text-to-domain datasets and 29K databases. Compared to the widely used Spider benchmark, we introduce a threefold increase in SQL patterns.
  arXiv  Detail & Related papers  (2023-05-25T17:19:52Z)
• Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs [89.68522473384522]
  We present Bird, a big benchmark for large-scale database grounded in text-to-efficient tasks. Our emphasis on database values highlights the new challenges of dirty database contents. Even the most effective text-to-efficient models, i.e. ChatGPT, achieves only 40.08% in execution accuracy.
  arXiv  Detail & Related papers  (2023-05-04T19:02:29Z)
• Towards Generalizable and Robust Text-to-SQL Parsing [77.18724939989647]
  We propose a novel TKK framework consisting of Task decomposition, Knowledge acquisition, and Knowledge composition to learn text-to- parsing in stages. We show that our framework is effective in all scenarios and state-of-the-art performance on the Spider, SParC, and Co. datasets.
  arXiv  Detail & Related papers  (2022-10-23T09:21:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.