Using LLM to select the right SQL Query from candidates

• URL: http://arxiv.org/abs/2401.02115v1
• Date: Thu, 4 Jan 2024 07:50:24 GMT
• Title: Using LLM to select the right SQL Query from candidates
• Abstract summary: We propose a re-rank method to select the right query from the candidate list. Experiment results show that the performance of some state-of-the-art models can get a 3.6% improvement after applying our re-rank method.
• Score: 7.462629286980003
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text-to-SQL models can generate a list of candidate SQL queries, and the best query is often in the candidate list, but not at the top of the list. An effective re-rank method can select the right SQL query from the candidate list and improve the model's performance. Previous studies on code generation automatically generate test cases and use them to re-rank candidate codes. However, automatic test case generation for text-to-SQL is an understudied field. We propose an automatic test case generation method that first generates a database and then uses LLMs to predict the ground truth, which is the expected execution results of the ground truth SQL query on this database. To reduce the difficulty for LLMs to predict, we conduct experiments to search for ways to generate easy databases for LLMs and design easy-to-understand prompts. Based on our test case generation method, we propose a re-rank method to select the right SQL query from the candidate list. Given a candidate list, our method can generate test cases and re-rank the candidate list according to their pass numbers on these test cases and their generation probabilities. The experiment results on the validation dataset of Spider show that the performance of some state-of-the-art models can get a 3.6\% improvement after applying our re-rank method.

Related papers

• Replacing Training with Memory: Listwise Selection for Text-to-SQL [34.25214572225323]
  We build reusable structured memories instead of learning selection behavior as model parameters.<n>We aggregate rankings across multiple input permutations, with inference cost optimized by execution results and pointwise scoring.<n>Across Text-to- benchmarks, it produces more stable selection without fine-tuning and fewer unnecessary comparisons than existing methods.
  arXiv  Detail & Related papers  (2026-09-01T07:35:00Z)
• CHS-SQL: A Text-to-SQL approach based on Confidence-Guided Heuristic Search Schema Linking process [1.4328341467798655]
  We propose a novel framework for efficiently fine-tuning SLMs on Text-to-SOTA tasks.<n>Its main innovation lies in the Linking phase, where a search combined with model internal confidence is employed.<n>Our method achieves state-of-the-art results on Text-to-SOTA tasks via SLMs.
  arXiv  Detail & Related papers  (2026-06-17T08:51:43Z)
• R$^3$-SQL: Ranking Reward and Resampling for Text-to-SQL [55.967050404665606]
  R$3$- achieves 75.03 execution accuracy on BIRD-dev, a new state of the art among methods using models with disclosed sizes, with consistent gains across five benchmarks.
  arXiv  Detail & Related papers  (2026-04-28T07:40:50Z)
• SQL-o1: A Self-Reward Heuristic Dynamic Search Method for Text-to-SQL [11.713258980098296]
  We propose a Self-Reward-based search method designed to enhance the reasoning ability of large language models (LLMs) Our code is publicly available at:https://github.com/ShuaiLyu01T-o1.
  arXiv  Detail & Related papers  (2025-02-17T12:28:11Z)
• Can the Rookies Cut the Tough Cookie? Exploring the Use of LLMs for SQL Equivalence Checking [15.42143912008553]
  We introduce a novel, realistic, and sufficiently complex benchmark called SQLEquiQuest for query equivalence checking.<n>We evaluate several state-of-the-art LLMs using various prompting strategies and carefully constructed in-context learning examples.<n>Our analysis shows that LLMs exhibit a strong bias for equivalence predictions, with consistently poor performance over non-equivalent pairs.
  arXiv  Detail & Related papers  (2024-12-07T06:50:12Z)
• CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQL [9.47170756607886]
  CHASE- is a new framework that employs innovative strategies, using test-time compute in multi-agent modeling to improve candidate generation and selection. To identify the best candidate, a selection agent is employed to rank the candidates through pairwise comparisons with a fine-tuned binary-candidates selection LLM. Overall, our proposed CHASE- achieves the state-of-the-art execution accuracy of 73.0% and 73.01% on the test set and development set of the notable BIRD Text-to- dataset benchmark.
  arXiv  Detail & Related papers  (2024-10-02T18:41:35Z)
• DAC: Decomposed Automation Correction for Text-to-SQL [51.48239006107272]
  We introduce De Automation Correction (DAC), which corrects text-to-composed by decomposing entity linking and skeleton parsing. We show that our method improves performance by $3.7%$ on average of Spider, Bird, and KaggleDBQA compared with the baseline method.
  arXiv  Detail & Related papers  (2024-08-16T14:43:15Z)
• RB-SQL: A Retrieval-based LLM Framework for Text-to-SQL [48.516004807486745]
  Large language models (LLMs) with in-context learning have significantly improved the performance of text-to- task. We propose RB-, a novel retrieval-based framework for in-context prompt engineering. Experiment results demonstrate that our model achieves better performance than several competitive baselines on public datasets BIRD and Spider.
  arXiv  Detail & Related papers  (2024-07-11T08:19:58Z)
• MCS-SQL: Leveraging Multiple Prompts and Multiple-Choice Selection For Text-to-SQL Generation [10.726734105960924]
  Large language models (LLMs) have enabled in-context learning (ICL)-based methods that significantly outperform fine-tuning approaches for text-to- tasks. This study considers the sensitivity of LLMs to the prompts and introduces a novel approach that leverages multiple prompts to explore a broader search space for possible answers. We establish a new SOTA performance on the BIRD in terms of both the accuracy and efficiency of the generated queries.
  arXiv  Detail & Related papers  (2024-05-13T04:59:32Z)
• LLM-R2: A Large Language Model Enhanced Rule-based Rewrite System for Boosting Query Efficiency [65.01402723259098]
  We propose a novel method of query rewrite named LLM-R2, adopting a large language model (LLM) to propose possible rewrite rules for a database rewrite system. Experimental results have shown that our method can significantly improve the query execution efficiency and outperform the baseline methods.
  arXiv  Detail & Related papers  (2024-04-19T13:17:07Z)
• SQLPrompt: In-Context Text-to-SQL with Minimal Labeled Data [54.69489315952524]
  "Prompt" is designed to improve the few-shot prompting capabilities of Text-to-LLMs. "Prompt" outperforms previous approaches for in-context learning with few labeled data by a large margin. We show that emphPrompt outperforms previous approaches for in-context learning with few labeled data by a large margin.
  arXiv  Detail & Related papers  (2023-11-06T05:24:06Z)
• Retrieval-augmented GPT-3.5-based Text-to-SQL Framework with Sample-aware Prompting and Dynamic Revision Chain [21.593701177605652]
  We propose a Text-to-aware prompting framework, involving a sample and a dynamic revision chain. Our approach incorporates sample demonstrations and fine-grained information related to the given question. To generate executable and accuratesqls without human intervention, we design a dynamic revision chain which iteratively adapts fine-grained feedback.
  arXiv  Detail & Related papers  (2023-07-11T07:16:22Z)
• Allies: Prompting Large Language Model with Beam Search [107.38790111856761]
  In this work, we propose a novel method called ALLIES. Given an input query, ALLIES leverages LLMs to iteratively generate new queries related to the original query. By iteratively refining and expanding the scope of the original query, ALLIES captures and utilizes hidden knowledge that may not be directly through retrieval.
  arXiv  Detail & Related papers  (2023-05-24T06:16:44Z)
• Large Language Models are Strong Zero-Shot Retriever [89.16756291653371]
  We propose a simple method that applies a large language model (LLM) to large-scale retrieval in zero-shot scenarios. Our method, the Language language model as Retriever (LameR), is built upon no other neural models but an LLM.
  arXiv  Detail & Related papers  (2023-04-27T14:45:55Z)
• T5QL: Taming language models for SQL generation [0.0]
  Current SOTA methods for semantic parsing depend on LLMs to achieve high predictive accuracy on benchmark datasets. We propose T5QL, a new method that improves the performance in benchmark datasets when using smaller LMs, namely T5-Base, by 13pp when compared against SOTA methods.
  arXiv  Detail & Related papers  (2022-09-21T10:43:13Z)
• Bertrand-DR: Improving Text-to-SQL using a Discriminative Re-ranker [1.049360126069332]
  We propose a novel discnative re-ranker to improve the performance of generative text-to-rimi models. We analyze relative strengths of the text-to-rimi and re-ranker models for optimal performance. We demonstrate the effectiveness of the re-ranker by applying it to two state-of-the-art text-to-rimi models.
  arXiv  Detail & Related papers  (2020-02-03T04:52:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.