ZAS-SQL: Distilling Rules from Failures for Zero-Shot Text-to-SQL

• URL: http://arxiv.org/abs/2606.08245v1
• Date: Sat, 06 Jun 2026 16:08:40 GMT
• Title: ZAS-SQL: Distilling Rules from Failures for Zero-Shot Text-to-SQL
• Abstract summary: Few-shot in-context learning methods built upon large language models (LLMs) achieve strong performance, yet their reliance on demonstrations limits cross-domain generalization.<n>We propose a fully zero-shot Text-to-Guided framework that distills core generation rules from failure cases through a Map-Reduce-based rule distillation pipeline.<n>The proposed framework achieves up to 87.2% and 88.6% execution accuracy on the Dev and Test sets, respectively, establishing a new zero-shot state-of-the-art.
• Score: 4.477529152940812
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text-to-SQL translates natural language into executable SQL queries. Few-shot in-context learning methods built upon large language models (LLMs) achieve strong performance, yet their reliance on demonstrations limits cross-domain generalization and consumes substantial context window space. Existing zero-shot methods, lacking effective generation constraints, still fall short of few-shot approaches. We observe that LLM failures in zero-shot Text-to-SQL are not random but exhibit systematic, recurring patterns. Building on this observation, we propose a fully zero-shot Text-to-SQL framework that distills core generation rules from failure cases through a Map-Reduce-based rule distillation pipeline and improves generation quality via three complementary modules: knowledge-augmented schema representation, which supplements missing semantics in Data Definition Language; a rule-driven structured reasoning framework that suppresses structural deviations; and Execution-Guided Early Stopping, which enables low-cost self-correction. On Spider, the proposed framework achieves up to 87.2% and 88.6% execution accuracy on the Dev and Test sets, respectively, establishing a new zero-shot state-of-the-art and surpassing multiple few-shot and fine-tuning methods built upon GPT-4/4o. On the domain-specific dataset UrbanPlan, it achieves 81.3%, confirming that the rule distillation approach generalizes across domains. Moreover, when equipped with a 4B-parameter model, the framework surpasses zero-shot baselines of leading closed-source models, demonstrating strong model generality.

Related papers

• ACTS-SQL: Agentic and Critic-Oriented Tree-Structured SQL Correctness with Large Language Models [27.75092433893197]
  We present a training-free framework that formulatessql correction as a plan-guided, tree-structured debug process.<n>We observe consistent accuracy gains over strong LLM backbones and representative agent-based baselines, achieving a 9.42% improvement over the previous state-of-the-art method.<n>The framework is also deployed in the Torch Log Service (TLS) of Volcano Engine to support an online Text-to-TLS API.
  arXiv  Detail & Related papers  (2026-08-15T09:43:57Z)
• Spider 2.0-AIFunc: Extending Real-World Text-to-SQL to AI-Native SQL Workflows [54.49251107006261]
  We introduce Spider 2.0-AIFunc, a benchmark of 465 verified instances across 125 real-world databases covering six types of AI functions on the Snowflake platform.<n>We find that the strongest proprietary models reach 67-70% execution accuracy while the best open-source model achieves 58.1%.
  arXiv  Detail & Related papers  (2026-07-07T12:54:08Z)
• Reliable Answers for Recurring Questions: Boosting Text-to-SQL Accuracy with Template Constrained Decoding [19.55574267306257]
  Large language models (LLMs) have revolutionized Text-to-generation, allowing users to query structured data using natural language with growing ease.<n>We introduce TeCoD, a system that harnesses the recurrence of query patterns in labeled workloads.<n>We show how TeCoD can yield up to 36% higher execution accuracy than in-context learning (ICL) and 2.2x lower latency on matched queries.
  arXiv  Detail & Related papers  (2026-04-30T15:44:34Z)
• VIRO: Robust and Efficient Neuro-Symbolic Reasoning with Verification for Referring Expression Comprehension [51.76841625486355]
  Referring Expression (REC) aims to localize the image region corresponding to a natural-language query.<n>Recent neuro-symbolic REC approaches leverage large language models (LLMs) and vision-language models (VLMs) to perform compositional reasoning.<n>We introduce VIRO, a neuro-symbolic framework that embeds lightweight operator-level verifiers within reasoning steps.
  arXiv  Detail & Related papers  (2026-01-19T07:21:19Z)
• Memo-SQL: Structured Decomposition and Experience-Driven Self-Correction for Training-Free NL2SQL [23.966546153810764]
  Existing NL2 systems rely on in-context learning with only correct examples.<n>We present Memo-correction, setting a new state of the art among open, zero-fine-tuning methods.
  arXiv  Detail & Related papers  (2026-01-15T02:42:05Z)
• Bridging Global Intent with Local Details: A Hierarchical Representation Approach for Semantic Validation in Text-to-SQL [30.78817492504152]
  HERO is a hierarchical representation approach that integrates global intent and local details.<n>We employ a Nested Message Passing Neural Network (NMPNN) to capture inherent information in relational schema-guided semantics.<n>Our approach outperforms existing state-of-the-art methods, achieving an average 9.40% improvement of AUPRC and 12.35% of AUROC in identifying semantic inconsistencies.<n>It excels at detecting fine-grained semantic errors, provides large language models with more granular feedback, and ultimately enhances the reliability and interpretability of data querying platforms.
  arXiv  Detail & Related papers  (2025-12-28T02:25:33Z)
• Text-to-SQL as Dual-State Reasoning: Integrating Adaptive Context and Progressive Generation [54.53145282349042]
  We introduce DSR-sourced, a textbfDual-textbfS textbfReasoning framework that models Text-to-context as an interaction between an adaptive context state and a progressive generation state.<n>Without any post-training or in-context examples, DSR-sourced achieves competitive performance, reaching 35.28% execution accuracy on Spider 2.0-Snow and 68.32% on BIRD development set.
  arXiv  Detail & Related papers  (2025-11-26T13:52:50Z)
• Evaluating NL2SQL via SQL2NL [45.88028371034407]
  New framework generates semantically equivalent, lexically diverse queries.<n>State-of-the-art models are far more brittle than standard benchmarks suggest.
  arXiv  Detail & Related papers  (2025-09-04T21:03:59Z)
• StarFT: Robust Fine-tuning of Zero-shot Models via Spuriosity Alignment [70.87096576708898]
  We propose StarFT, a framework for fine-tuning zero-shot models to enhance robustness by preventing them from learning spuriosity.<n>StarFT boosts both worst-group and average accuracy by 14.30% and 3.02%, respectively, in the Waterbirds group shift scenario.
  arXiv  Detail & Related papers  (2025-05-19T15:15:35Z)
• OpenSearch-SQL: Enhancing Text-to-SQL with Dynamic Few-shot and Consistency Alignment [6.2089733671434875]
  We propose OpenSearch-, which divides the Text-to-agent task into four main modules: Preprocessing, Extraction, Generation, and Refinement, along with an Alignment module based on consistency alignment mechanism.<n>These methods have significantly improved the performance of LLMs in the Text-to-agent task.<n> Experimental results show that OpenSearch- achieves an execution accuracy(EX) of 69.3% on the BIRD development set, 72.28% on the test set, and a reward-based efficiency score (R-VES) of 69.3, with all three metrics ranking first at the time of submission.
  arXiv  Detail & Related papers  (2025-02-19T07:51:50Z)
• RSL-SQL: Robust Schema Linking in Text-to-SQL Generation [51.00761167842468]
  We propose a novel framework called RSL- that combines bidirectional schema linking, contextual information augmentation, binary selection strategy, and multi-turn self-correction. benchmarks demonstrate that our approach achieves SOTA execution accuracy among open-source solutions, with 67.2% on BIRD and 87.9% on GPT-4ocorrection. Our approach outperforms a series of GPT-4 based Text-to-Seek systems when adopting DeepSeek (much cheaper) with same intact prompts.
  arXiv  Detail & Related papers  (2024-10-31T16:22:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.