SLM-SQL: An Exploration of Small Language Models for Text-to-SQL

• URL: http://arxiv.org/abs/2507.22478v1
• Date: Wed, 30 Jul 2025 08:29:07 GMT
• Title: SLM-SQL: An Exploration of Small Language Models for Text-to-SQL
• Authors: Lei Sheng, Shuai-Shuai Xu,
• Abstract summary: Small language models (SLMs) offer inherent advantages in inference speed and suitability for edge deployment.<n>We leverage recent advancements in post-training techniques to explore Textto- applications.<n> Experimental results validate the effectiveness and generalizability of our method.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) have demonstrated strong performance in translating natural language questions into SQL queries (Text-to-SQL). In contrast, small language models (SLMs) ranging from 0.5B to 1.5B parameters currently underperform on Text-to-SQL tasks due to their limited logical reasoning capabilities. However, SLMs offer inherent advantages in inference speed and suitability for edge deployment. To explore their potential in Text-to-SQL applications, we leverage recent advancements in post-training techniques. Specifically, we used the open-source SynSQL-2.5M dataset to construct two derived datasets: SynSQL-Think-916K for SQL generation and SynSQL-Merge-Think-310K for SQL merge revision. We then applied supervised fine-tuning and reinforcement learning-based post-training to the SLM, followed by inference using a corrective self-consistency approach. Experimental results validate the effectiveness and generalizability of our method, SLM-SQL. On the BIRD development set, the five evaluated models achieved an average improvement of 31.4 points. Notably, the 0.5B model reached 56.87\% execution accuracy (EX), while the 1.5B model achieved 67.08\% EX. We will release our dataset, model, and code to github: https://github.com/CycloneBoy/slm_sql.

Related papers

• CogniSQL-R1-Zero: Lightweight Reinforced Reasoning for Efficient SQL Generation [1.169202600932732]
  We introduce Cogni-R1-Zero, a reinforcement learning (RL) framework and model.<n>We use a lightweight reward signal based on execution correctness and format-tag compliance.<n>Our method achieves state-of-the-art execution accuracy on Text2 benchmark.<n>To support further research in efficient and interpretable Text-to-code modeling, we release two curated datasets.
  arXiv  Detail & Related papers  (2025-07-08T14:17:07Z)
• SDE-SQL: Enhancing Text-to-SQL Generation in Large Language Models via Self-Driven Exploration with SQL Probes [10.672822970425404]
  We propose SDE-, a framework that enables large language models to perform self-driven exploration of databases during inference.<n>Unlike prior methods, SDE- operates in a zero-shot setting, without relying on any question-context pairs as in-context demonstrations.
  arXiv  Detail & Related papers  (2025-06-08T18:01:26Z)
• SQLForge: Synthesizing Reliable and Diverse Data to Enhance Text-to-SQL Reasoning in LLMs [17.91039683592367]
  Large Language models (LLMs) have demonstrated significant potential in text-to- template reasoning tasks.<n>Yet a substantial performance gap persists between open-source models and their closed-source counterparts.<n>We introduce SQLForge, a novel approach for synthesizing reliable and diverse data.
  arXiv  Detail & Related papers  (2025-05-19T20:51:29Z)
• ExCoT: Optimizing Reasoning for Text-to-SQL with Execution Feedback [49.21833666405111]
  Large language models (LLMs) excel in many reasoning tasks, but their ability to leverage Chain-of-Thought (CoT) reasoning remains underexplored.<n>We propose ExCoT, a novel framework that iteratively optimize open-source LLMs by combining CoT reasoning with off-policy and on-policy DPO.
  arXiv  Detail & Related papers  (2025-03-25T18:17:36Z)
• Feather-SQL: A Lightweight NL2SQL Framework with Dual-Model Collaboration Paradigm for Small Language Models [22.960560371494832]
  Small language models (SLMs) struggle with NL2 tasks, exhibiting poor performance and incompatibility with existing frameworks.<n>We introduce Feather- Paradigm, a new lightweight framework tailored for SLMs.<n>The proposed paradigm raises the accuracy ceiling of SLMs to 54.76%, highlighting its effectiveness.
  arXiv  Detail & Related papers  (2025-03-22T16:22:53Z)
• MCTS-SQL: Light-Weight LLMs can Master the Text-to-SQL through Monte Carlo Tree Search [1.166711394125328]
  Text-to-OTA is a fundamental yet challenging task in the NLP area.<n>We propose MCTS-OTA, a novel framework that uses Monte Carlo Tree Search.<n>We propose a token-level prefixcache mechanism that stores prior information during iterations.
  arXiv  Detail & Related papers  (2025-01-28T00:52:23Z)
• RSL-SQL: Robust Schema Linking in Text-to-SQL Generation [51.00761167842468]
  We propose a novel framework called RSL- that combines bidirectional schema linking, contextual information augmentation, binary selection strategy, and multi-turn self-correction. benchmarks demonstrate that our approach achieves SOTA execution accuracy among open-source solutions, with 67.2% on BIRD and 87.9% on GPT-4ocorrection. Our approach outperforms a series of GPT-4 based Text-to-Seek systems when adopting DeepSeek (much cheaper) with same intact prompts.
  arXiv  Detail & Related papers  (2024-10-31T16:22:26Z)
• SQLPrompt: In-Context Text-to-SQL with Minimal Labeled Data [54.69489315952524]
  "Prompt" is designed to improve the few-shot prompting capabilities of Text-to-LLMs. "Prompt" outperforms previous approaches for in-context learning with few labeled data by a large margin. We show that emphPrompt outperforms previous approaches for in-context learning with few labeled data by a large margin.
  arXiv  Detail & Related papers  (2023-11-06T05:24:06Z)
• SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL (extended) [53.95151604061761]
  This paper introduces the framework for enhancing Text-to- filtering using large language models (LLMs) With few-shot prompting, we explore the effectiveness of consistency decoding with execution-based error analyses. With instruction fine-tuning, we delve deep in understanding the critical paradigms that influence the performance of tuned LLMs.
  arXiv  Detail & Related papers  (2023-05-26T21:39:05Z)
• UNITE: A Unified Benchmark for Text-to-SQL Evaluation [72.72040379293718]
  We introduce a UNIfied benchmark for Text-to-domain systems. It is composed of publicly available text-to-domain datasets and 29K databases. Compared to the widely used Spider benchmark, we introduce a threefold increase in SQL patterns.
  arXiv  Detail & Related papers  (2023-05-25T17:19:52Z)
• Weakly Supervised Text-to-SQL Parsing through Question Decomposition [53.22128541030441]
  We take advantage of the recently proposed question meaning representation called QDMR. Given questions, their QDMR structures (annotated by non-experts or automatically predicted) and the answers, we are able to automatically synthesizesql queries. Our results show that the weakly supervised models perform competitively with those trained on NL- benchmark data.
  arXiv  Detail & Related papers  (2021-12-12T20:02:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.