UniQL: Towards Dialect-Universal Benchmarking for Text-to-SQL

• URL: http://arxiv.org/abs/2606.08018v1
• Date: Sat, 06 Jun 2026 07:14:53 GMT
• Title: UniQL: Towards Dialect-Universal Benchmarking for Text-to-SQL
• Abstract summary: We introduce UniQL, a human-verified benchmark for cross-dialect text-to- evaluation.<n>All dialects share the same intents, aligned schemas and database contents, enabling controlled evaluation of dialect generalization.
• Score: 27.963417399820987
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing text-to-SQL benchmarks are largely centered on SQLite, making it difficult to evaluate whether models can generalize across heterogeneous SQL dialects. However, real-world database systems differ substantially in syntax, functions, type systems, and execution semantics, so the same natural language intent often requires dialect-specific SQL realizations. We introduce UniQL, a human-verified benchmark for cross-dialect text-to-SQL evaluation. UniQL aligns 1,534 natural language questions with executable SQL annotations across 16 SQL dialects, yielding 24,544 dialect-specific queries. All dialects share the same intents, aligned schemas and database contents, enabling controlled evaluation of dialect generalization. UniQL is constructed through a hybrid pipeline combining database migration, SQL translation, execution-guided verification, iterative rule summarization, and human validation. Experiments on both open-source and closed-source LLMs show that current models remain far from dialect-universal, with substantial performance variation across database systems and limited transfer from SQLite success to other dialects. These findings highlight the need for aligned cross-dialect benchmarks and more dialect-aware text-to-SQL methods. Code and data are available at https://github.com/JerryGao818/UniQL

Related papers

• ProSPy: A Profiling-Driven SQL-Python Agentic Framework for Enterprise Text-to-SQL [51.98164069124653]
  ProSPy is a Profiling-driven framework for enterprise-scale Text-to-gnostic analysis.<n>ProSPy structures the reasoning process into four stages.<n>It first extracts fine-grained data evidence through automatic profiling.<n>It prunes large schemas into task-relevant contexts.
  arXiv  Detail & Related papers  (2026-06-04T08:13:05Z)
• PolySQL: Scaling Text-to-SQL Evaluation Across SQL Dialects via Automated Backend Isomorphism [5.75736217540603]
  Polylect is a novel dual-exebust method that eliminates the need for query transpilation by comparing normalized execution results.<n>Our study reveals a 10.1% average accuracy drop from other dialects and identifies a significant dialect difficulty hierarchy.
  arXiv  Detail & Related papers  (2026-05-08T14:32:44Z)
• Dialect-Agnostic SQL Parsing via LLM-Based Segmentation [7.41731253317229]
  We propose a novel query rewriting framework that integrates grammar-based parsing with LLM-based segmentation to parse dialects robustly.<n>Our core idea is to decompose hierarchical parsing to sequential segmentation tasks, which better aligns with the strength of LLMs.<n>In a standalone evaluation, it parses 91.55% to 100% of queries across eight distinct dialects, outperforming all baselines.
  arXiv  Detail & Related papers  (2026-03-17T06:18:37Z)
• PARROT: A Benchmark for Evaluating LLMs in Cross-System SQL Translation [21.0303026118673]
  We introduce PARROT, a practical and realistic benchmak for CrOss-System SQL Translation.<n> PARROT comprises 598 translation pairs from 38 open-source benchmarks and real-world business services.<n>We also provide multiple benchmark variants, including PARROT-Diverse with 28,003 translations and PARROT-Simple with 5,306 representative samples.
  arXiv  Detail & Related papers  (2025-09-27T14:41:13Z)
• ExeSQL: Self-Taught Text-to-SQL Models with Execution-Driven Bootstrapping for SQL Dialects [24.450818792474216]
  This work introduces Exe, a text-to-guided framework with execution-driven, agentic bootstrapping.<n>We show that Exe bridges the dialect gap in text-to-guided learning, achieving average improvements of 15.2%, 10.38%, and 4.49% over GPT-4o on, and Oracle, respectively.
  arXiv  Detail & Related papers  (2025-05-22T19:13:34Z)
• SQL-GEN: Bridging the Dialect Gap for Text-to-SQL Via Synthetic Data And Model Merging [30.306023265985658]
  We introduce a framework for generating high-quality synthetic training data for any dialect. We propose a novel Mixture-of-Experts (MoE) that leverages the shared knowledge across dialects.
  arXiv  Detail & Related papers  (2024-08-22T20:50:48Z)
• SQLPrompt: In-Context Text-to-SQL with Minimal Labeled Data [54.69489315952524]
  "Prompt" is designed to improve the few-shot prompting capabilities of Text-to-LLMs. "Prompt" outperforms previous approaches for in-context learning with few labeled data by a large margin. We show that emphPrompt outperforms previous approaches for in-context learning with few labeled data by a large margin.
  arXiv  Detail & Related papers  (2023-11-06T05:24:06Z)
• UNITE: A Unified Benchmark for Text-to-SQL Evaluation [72.72040379293718]
  We introduce a UNIfied benchmark for Text-to-domain systems. It is composed of publicly available text-to-domain datasets and 29K databases. Compared to the widely used Spider benchmark, we introduce a threefold increase in SQL patterns.
  arXiv  Detail & Related papers  (2023-05-25T17:19:52Z)
• XRICL: Cross-lingual Retrieval-Augmented In-Context Learning for Cross-lingual Text-to-SQL Semantic Parsing [70.40401197026925]
  In-context learning using large language models has recently shown surprising results for semantic parsing tasks. This work introduces the XRICL framework, which learns to retrieve relevant English exemplars for a given query. We also include global translation exemplars for a target language to facilitate the translation process for large language models.
  arXiv  Detail & Related papers  (2022-10-25T01:33:49Z)
• A Survey on Text-to-SQL Parsing: Concepts, Methods, and Future Directions [102.8606542189429]
  The goal of text-to-corpora parsing is to convert a natural language (NL) question to its corresponding structured query language () based on the evidences provided by databases. Deep neural networks have significantly advanced this task by neural generation models, which automatically learn a mapping function from an input NL question to an output query.
  arXiv  Detail & Related papers  (2022-08-29T14:24:13Z)
• Weakly Supervised Text-to-SQL Parsing through Question Decomposition [53.22128541030441]
  We take advantage of the recently proposed question meaning representation called QDMR. Given questions, their QDMR structures (annotated by non-experts or automatically predicted) and the answers, we are able to automatically synthesizesql queries. Our results show that the weakly supervised models perform competitively with those trained on NL- benchmark data.
  arXiv  Detail & Related papers  (2021-12-12T20:02:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.