SQLCritic: Correcting Text-to-SQL Generation via Clause-wise Critic

• URL: http://arxiv.org/abs/2503.07996v4
• Date: Wed, 21 May 2025 01:48:22 GMT
• Title: SQLCritic: Correcting Text-to-SQL Generation via Clause-wise Critic
• Authors: Jikai Chen, Leilei Gan, Ziyu Zhao, Zechuan Wang, Dong Wang, Chenyi Zhuang,
• Abstract summary: We introduce a clause-wise critique generation task along with a benchmark,sqlCriticBench, which performs fine-grained error localization.<n>We also propose an automatically training dataset curation pipeline which annotates clause-wise critique at scale.
• Score: 8.680252929322684
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing refinement methods in LLM-based Text-to-SQL systems exhibit limited effectiveness. They often introduce new errors during the self-correction process and fail to detect and correct semantic inaccuracies. To address these gaps, we first introduce a clause-wise critique generation task along with a benchmark, SQLCriticBench, which performs fine-grained error localization including both syntax and semantic errors at the clause level. Furthermore, we introduce a variant of DPO for training our SQLCritic model, where the $\beta$ coefficient is adaptively changed according to the clause-level inconsistencies between the preferred and dispreferred critiques. We also propose an automatically training dataset curation pipeline which annotate clause-wise critique at scale in a cost-effective way. Experiments demonstrate that the SQLCritic model significantly improves SQL accuracy on the BIRD and Spider datasets, and the results on SQLCriticBench further reveals its superior critique capabilities compared to existing models.

Related papers

• RetrySQL: text-to-SQL training with retry data for self-correcting query generation [1.6707278580444538]
  We introduce Retry, a new approach to training text-to-generation models.<n>We demonstrate that retry steps yield an improvement of up to 4 percentage points in both overall and challenging execution accuracy metrics.
  arXiv  Detail & Related papers  (2025-07-03T11:00:49Z)
• Calibrating LLMs for Text-to-SQL Parsing by Leveraging Sub-clause Frequencies [28.281517110365037]
  We study the problem of providing a calibrated confidence score that conveys the likelihood of an output query being correct.<n>Our work is the first to establish a benchmark for post-hoc calibration of text-to- parsing.
  arXiv  Detail & Related papers  (2025-05-27T01:01:55Z)
• CSC-SQL: Corrective Self-Consistency in Text-to-SQL via Reinforcement Learning [0.0]
  We propose CSC-, a novel method that integrates Self-Consistency and Self-Correction.<n>We employ the Group Relative Policy (GRPO) algorithm to fine-tune both the SQL generation and revision models.<n>On the BIRD private test set, our 7B model achieves 71.72% execution accuracy, while the 32B model 73.67%.
  arXiv  Detail & Related papers  (2025-05-19T15:52:19Z)
• Learning from Imperfect Data: Towards Efficient Knowledge Distillation of Autoregressive Language Models for Text-to-SQL [83.99974309930072]
  Knowledge distillation (KD) is a common approach, which aims to distill the larger teacher model into a smaller student model. We propose to improve the KD with Imperfect Data, namely KID, which effectively boosts the performance without introducing much training budget. KID can not only achieve consistent and significant performance gains across all model types and sizes, but also effectively improve the training efficiency.
  arXiv  Detail & Related papers  (2024-10-15T07:51:00Z)
• Context-Aware SQL Error Correction Using Few-Shot Learning -- A Novel Approach Based on NLQ, Error, and SQL Similarity [0.0]
  This paper introduces a novel few-shot learning-based approach for error correction insql generation. It enhances the accuracy of generated queries by selecting the most suitable few-shot error correction examples for a given natural language question (NLQ) In experiments with the open-source dataset, the proposed model offers a 39.2% increase in fixing errors with no error correction and a 10% increase from a simple error correction method.
  arXiv  Detail & Related papers  (2024-10-11T18:22:08Z)
• DAC: Decomposed Automation Correction for Text-to-SQL [51.48239006107272]
  We introduce De Automation Correction (DAC), which corrects text-to-composed by decomposing entity linking and skeleton parsing. We show that our method improves performance by $3.7%$ on average of Spider, Bird, and KaggleDBQA compared with the baseline method.
  arXiv  Detail & Related papers  (2024-08-16T14:43:15Z)
• ETM: Modern Insights into Perspective on Text-to-SQL Evaluation in the Age of Large Language Models [8.618945530676614]
  Execution Accuracy (EXE) and Exact Set Matching Accuracy (ESM) suffer from inherent limitations that can misrepresent performance.<n>We introduce a new metric, Enhanced Tree Matching (ETM), which mitigates these issues by comparing queries using both syntactic and semantic elements.<n>We show that ETM and ESM can produce false positive and negative rates as high as 23.0% and 28.9%, while ETM reduces these rates to 0.3% and 2.7%, respectively.
  arXiv  Detail & Related papers  (2024-07-10T02:20:19Z)
• Correcting Semantic Parses with Natural Language through Dynamic Schema Encoding [0.06445605125467573]
  We show that the accuracy of autoregressive decoders can be boosted by up to 26% with only one turn of correction with natural language. A Tbase model is capable of correcting the errors of a T5-large model in a zero-shot, cross-parser setting.
  arXiv  Detail & Related papers  (2023-05-31T16:01:57Z)
• SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL (extended) [53.95151604061761]
  This paper introduces the framework for enhancing Text-to- filtering using large language models (LLMs) With few-shot prompting, we explore the effectiveness of consistency decoding with execution-based error analyses. With instruction fine-tuning, we delve deep in understanding the critical paradigms that influence the performance of tuned LLMs.
  arXiv  Detail & Related papers  (2023-05-26T21:39:05Z)
• Error Detection for Text-to-SQL Semantic Parsing [18.068244400731366]
  Modern text-to- semantics are often over-confident, casting doubt on their trustworthiness when deployed for real use. We propose a-independent error detection model for text-to- semantic parsing.
  arXiv  Detail & Related papers  (2023-05-23T04:44:22Z)
• Wav2SQL: Direct Generalizable Speech-To-SQL Parsing [55.10009651476589]
  Speech-to-Spider (S2Spider) aims to convert spoken questions intosql queries given databases. We propose the first direct speech-to-speaker parsing model Wav2 which avoids error compounding across cascaded systems. Experimental results demonstrate that Wav2 avoids error compounding and achieves state-of-the-art results by up to 2.5% accuracy improvement over the baseline.
  arXiv  Detail & Related papers  (2023-05-21T19:26:46Z)
• SUN: Exploring Intrinsic Uncertainties in Text-to-SQL Parsers [61.48159785138462]
  This paper aims to improve the performance of text-to-dependence by exploring the intrinsic uncertainties in the neural network based approaches (called SUN) Extensive experiments on five benchmark datasets demonstrate that our method significantly outperforms competitors and achieves new state-of-the-art results.
  arXiv  Detail & Related papers  (2022-09-14T06:27:51Z)
• S$^2$SQL: Injecting Syntax to Question-Schema Interaction Graph Encoder for Text-to-SQL Parsers [66.78665327694625]
  We propose S$2$, injecting Syntax to question- encoder graph for Text-to- relational parsing. We also employ the decoupling constraint to induce diverse edge embedding, which further improves the network's performance. Experiments on the Spider and robustness setting Spider-Syn demonstrate that the proposed approach outperforms all existing methods when pre-training models are used.
  arXiv  Detail & Related papers  (2022-03-14T09:49:15Z)
• Tail-to-Tail Non-Autoregressive Sequence Prediction for Chinese Grammatical Error Correction [49.25830718574892]
  We present a new framework named Tail-to-Tail (textbfTtT) non-autoregressive sequence prediction. Considering that most tokens are correct and can be conveyed directly from source to target, and the error positions can be estimated and corrected. Experimental results on standard datasets, especially on the variable-length datasets, demonstrate the effectiveness of TtT in terms of sentence-level Accuracy, Precision, Recall, and F1-Measure.
  arXiv  Detail & Related papers  (2021-06-03T05:56:57Z)
• Photon: A Robust Cross-Domain Text-to-SQL System [189.1405317853752]
  We present Photon, a robust, modular, cross-domain NLIDB that can flag natural language input to which a mapping cannot be immediately determined. The proposed method effectively improves the robustness of text-to-native system against untranslatable user input.
  arXiv  Detail & Related papers  (2020-07-30T07:44:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.