MSc-SQL: Multi-Sample Critiquing Small Language Models For Text-To-SQL Translation

• URL: http://arxiv.org/abs/2410.12916v1
• Date: Wed, 16 Oct 2024 18:03:24 GMT
• Title: MSc-SQL: Multi-Sample Critiquing Small Language Models For Text-To-SQL Translation
• Authors: Satya Krishna Gorti, Ilan Gofman, Zhaoyan Liu, Jiapeng Wu, Noël Vouitsis, Guangwei Yu, Jesse C. Cresswell, Rasa Hosseinzadeh,
• Abstract summary: Text-to-generation enables non-experts to interact with databases via natural language. Recent advances on large closed-source models like GPT-4 present challenges in accessibility, privacy, and latency. We focus on developing small, efficient, and open-source text-to-generation models.
• Score: 10.205010004198757
• License:
• Abstract: Text-to-SQL generation enables non-experts to interact with databases via natural language. Recent advances rely on large closed-source models like GPT-4 that present challenges in accessibility, privacy, and latency. To address these issues, we focus on developing small, efficient, and open-source text-to-SQL models. We demonstrate the benefits of sampling multiple candidate SQL generations and propose our method, MSc-SQL, to critique them using associated metadata. Our sample critiquing model evaluates multiple outputs simultaneously, achieving state-of-the-art performance compared to other open-source models while remaining competitive with larger models at a much lower cost. Full code can be found at github.com/layer6ai-labs/msc-sql.

Related papers

• DataGpt-SQL-7B: An Open-Source Language Model for Text-to-SQL [7.76068876576964]
  We propose a suite of compact, fine-tuned models and self-refine mechanisms to democratize data access and analysis for non-expert users. Our system, DataGpt-sql, achieved 87.2% accuracy on the spider-dev.
  arXiv  Detail & Related papers  (2024-09-24T11:38:08Z)
• SQL-GEN: Bridging the Dialect Gap for Text-to-SQL Via Synthetic Data And Model Merging [30.306023265985658]
  We introduce a framework for generating high-quality synthetic training data for any dialect. We propose a novel Mixture-of-Experts (MoE) that leverages the shared knowledge across dialects.
  arXiv  Detail & Related papers  (2024-08-22T20:50:48Z)
• Synthesizing Text-to-SQL Data from Weak and Strong LLMs [68.69270834311259]
  The capability gap between open-source and closed-source large language models (LLMs) remains a challenge in text-to- tasks. We introduce a synthetic data approach that combines data produced by larger, more powerful models with error information data generated by smaller, not well-aligned models.
  arXiv  Detail & Related papers  (2024-08-06T15:40:32Z)
• TrustSQL: Benchmarking Text-to-SQL Reliability with Penalty-Based Scoring [11.78795632771211]
  We introduce a novel benchmark designed to evaluate text-to- reliability as a model's ability to correctly handle any type of input question. We evaluate existing methods using a novel penalty-based scoring metric with two modeling approaches.
  arXiv  Detail & Related papers  (2024-03-23T16:12:52Z)
• SQLPrompt: In-Context Text-to-SQL with Minimal Labeled Data [54.69489315952524]
  "Prompt" is designed to improve the few-shot prompting capabilities of Text-to-LLMs. "Prompt" outperforms previous approaches for in-context learning with few labeled data by a large margin. We show that emphPrompt outperforms previous approaches for in-context learning with few labeled data by a large margin.
  arXiv  Detail & Related papers  (2023-11-06T05:24:06Z)
• Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation [76.76046657162306]
  Large language models (LLMs) have emerged as a new paradigm for Text-to- task. Large language models (LLMs) have emerged as a new paradigm for Text-to- task.
  arXiv  Detail & Related papers  (2023-08-29T14:59:54Z)
• Natural language to SQL in low-code platforms [0.0]
  We propose a pipeline allowing developers to write natural language (NL) queries. We collect, label, and validate data covering the queries most often performed by OutSystems users. We describe the entire pipeline, which comprises a feedback loop that allows us to quickly collect production data.
  arXiv  Detail & Related papers  (2023-08-29T11:59:02Z)
• SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL (extended) [53.95151604061761]
  This paper introduces the framework for enhancing Text-to- filtering using large language models (LLMs) With few-shot prompting, we explore the effectiveness of consistency decoding with execution-based error analyses. With instruction fine-tuning, we delve deep in understanding the critical paradigms that influence the performance of tuned LLMs.
  arXiv  Detail & Related papers  (2023-05-26T21:39:05Z)
• XRICL: Cross-lingual Retrieval-Augmented In-Context Learning for Cross-lingual Text-to-SQL Semantic Parsing [70.40401197026925]
  In-context learning using large language models has recently shown surprising results for semantic parsing tasks. This work introduces the XRICL framework, which learns to retrieve relevant English exemplars for a given query. We also include global translation exemplars for a target language to facilitate the translation process for large language models.
  arXiv  Detail & Related papers  (2022-10-25T01:33:49Z)
• Weakly Supervised Text-to-SQL Parsing through Question Decomposition [53.22128541030441]
  We take advantage of the recently proposed question meaning representation called QDMR. Given questions, their QDMR structures (annotated by non-experts or automatically predicted) and the answers, we are able to automatically synthesizesql queries. Our results show that the weakly supervised models perform competitively with those trained on NL- benchmark data.
  arXiv  Detail & Related papers  (2021-12-12T20:02:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.