SLM-SQL: An Exploration of Small Language Models for Text-to-SQL

• URL: http://arxiv.org/abs/2507.22478v1
• Date: Wed, 30 Jul 2025 08:29:07 GMT
• Title: SLM-SQL: An Exploration of Small Language Models for Text-to-SQL
• Abstract summary: Small language models (SLMs) offer inherent advantages in inference speed and suitability for edge deployment.<n>We leverage recent advancements in post-training techniques to explore Textto- applications.<n> Experimental results validate the effectiveness and generalizability of our method.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) have demonstrated strong performance in translating natural language questions into SQL queries (Text-to-SQL). In contrast, small language models (SLMs) ranging from 0.5B to 1.5B parameters currently underperform on Text-to-SQL tasks due to their limited logical reasoning capabilities. However, SLMs offer inherent advantages in inference speed and suitability for edge deployment. To explore their potential in Text-to-SQL applications, we leverage recent advancements in post-training techniques. Specifically, we used the open-source SynSQL-2.5M dataset to construct two derived datasets: SynSQL-Think-916K for SQL generation and SynSQL-Merge-Think-310K for SQL merge revision. We then applied supervised fine-tuning and reinforcement learning-based post-training to the SLM, followed by inference using a corrective self-consistency approach. Experimental results validate the effectiveness and generalizability of our method, SLM-SQL. On the BIRD development set, the five evaluated models achieved an average improvement of 31.4 points. Notably, the 0.5B model reached 56.87\% execution accuracy (EX), while the 1.5B model achieved 67.08\% EX. We will release our dataset, model, and code to github: https://github.com/CycloneBoy/slm_sql.

Related papers

• Spider 2.0-AIFunc: Extending Real-World Text-to-SQL to AI-Native SQL Workflows [54.49251107006261]
  We introduce Spider 2.0-AIFunc, a benchmark of 465 verified instances across 125 real-world databases covering six types of AI functions on the Snowflake platform.<n>We find that the strongest proprietary models reach 67-70% execution accuracy while the best open-source model achieves 58.1%.
  arXiv  Detail & Related papers  (2026-07-07T12:54:08Z)
• FINER-SQL: Boosting Small Language Models for Text-to-SQL [40.19592881059662]
  Large language models suffer from high computational cost, long latency, and data privacy concerns.<n>A natural alternative is to use small language models, which enable efficient and private on-premise deployment.<n>We propose FINER-sql, a scalable and reusable reinforcement learning framework.
  arXiv  Detail & Related papers  (2026-05-05T07:51:29Z)
• Track-SQL: Enhancing Generative Language Models with Dual-Extractive Modules for Schema and Context Tracking in Multi-turn Text-to-SQL [50.84251174559123]
  Generative language models' performance does not extend equivalently to multi-turn Text-to-sourced.<n>We propose Track-, which enhances generative language models with dual-extractive modules designed to track schema and contextual changes in multi-turn Text-to-sourced.<n>Track- significantly improves execution accuracy in multi-turn interactions by 7.1% and 9.55% on datasets, respectively.
  arXiv  Detail & Related papers  (2026-03-06T07:57:06Z)
• Text-to-SQL as Dual-State Reasoning: Integrating Adaptive Context and Progressive Generation [54.53145282349042]
  We introduce DSR-sourced, a textbfDual-textbfS textbfReasoning framework that models Text-to-context as an interaction between an adaptive context state and a progressive generation state.<n>Without any post-training or in-context examples, DSR-sourced achieves competitive performance, reaching 35.28% execution accuracy on Spider 2.0-Snow and 68.32% on BIRD development set.
  arXiv  Detail & Related papers  (2025-11-26T13:52:50Z)
• DeKeyNLU: Enhancing Natural Language to SQL Generation through Task Decomposition and Keyword Extraction [46.422626657078666]
  We present DeKeyNLU, a novel dataset which contains 1,500 meticulously annotated QA pairs.<n>We propose DeKey, a RAG-based NL2 pipeline that employs three separate modules for user question understanding, entity retrieval, and generation.
  arXiv  Detail & Related papers  (2025-09-18T00:47:56Z)
• CogniSQL-R1-Zero: Lightweight Reinforced Reasoning for Efficient SQL Generation [1.169202600932732]
  We introduce Cogni-R1-Zero, a reinforcement learning (RL) framework and model.<n>We use a lightweight reward signal based on execution correctness and format-tag compliance.<n>Our method achieves state-of-the-art execution accuracy on Text2 benchmark.<n>To support further research in efficient and interpretable Text-to-code modeling, we release two curated datasets.
  arXiv  Detail & Related papers  (2025-07-08T14:17:07Z)
• SDE-SQL: Enhancing Text-to-SQL Generation in Large Language Models via Self-Driven Exploration with SQL Probes [10.672822970425404]
  We propose SDE-, a framework that enables large language models to perform self-driven exploration of databases during inference.<n>Unlike prior methods, SDE- operates in a zero-shot setting, without relying on any question-context pairs as in-context demonstrations.
  arXiv  Detail & Related papers  (2025-06-08T18:01:26Z)
• SQLForge: Synthesizing Reliable and Diverse Data to Enhance Text-to-SQL Reasoning in LLMs [17.91039683592367]
  Large Language models (LLMs) have demonstrated significant potential in text-to- template reasoning tasks.<n>Yet a substantial performance gap persists between open-source models and their closed-source counterparts.<n>We introduce SQLForge, a novel approach for synthesizing reliable and diverse data.
  arXiv  Detail & Related papers  (2025-05-19T20:51:29Z)
• ExCoT: Optimizing Reasoning for Text-to-SQL with Execution Feedback [49.21833666405111]
  Large language models (LLMs) excel in many reasoning tasks, but their ability to leverage Chain-of-Thought (CoT) reasoning remains underexplored.<n>We propose ExCoT, a novel framework that iteratively optimize open-source LLMs by combining CoT reasoning with off-policy and on-policy DPO.
  arXiv  Detail & Related papers  (2025-03-25T18:17:36Z)
• Feather-SQL: A Lightweight NL2SQL Framework with Dual-Model Collaboration Paradigm for Small Language Models [22.960560371494832]
  Small language models (SLMs) struggle with NL2 tasks, exhibiting poor performance and incompatibility with existing frameworks.<n>We introduce Feather- Paradigm, a new lightweight framework tailored for SLMs.<n>The proposed paradigm raises the accuracy ceiling of SLMs to 54.76%, highlighting its effectiveness.
  arXiv  Detail & Related papers  (2025-03-22T16:22:53Z)
• MCTS-SQL: Light-Weight LLMs can Master the Text-to-SQL through Monte Carlo Tree Search [1.166711394125328]
  Text-to-OTA is a fundamental yet challenging task in the NLP area.<n>We propose MCTS-OTA, a novel framework that uses Monte Carlo Tree Search.<n>We propose a token-level prefixcache mechanism that stores prior information during iterations.
  arXiv  Detail & Related papers  (2025-01-28T00:52:23Z)
• RSL-SQL: Robust Schema Linking in Text-to-SQL Generation [51.00761167842468]
  We propose a novel framework called RSL- that combines bidirectional schema linking, contextual information augmentation, binary selection strategy, and multi-turn self-correction. benchmarks demonstrate that our approach achieves SOTA execution accuracy among open-source solutions, with 67.2% on BIRD and 87.9% on GPT-4ocorrection. Our approach outperforms a series of GPT-4 based Text-to-Seek systems when adopting DeepSeek (much cheaper) with same intact prompts.
  arXiv  Detail & Related papers  (2024-10-31T16:22:26Z)
• SQLPrompt: In-Context Text-to-SQL with Minimal Labeled Data [54.69489315952524]
  "Prompt" is designed to improve the few-shot prompting capabilities of Text-to-LLMs. "Prompt" outperforms previous approaches for in-context learning with few labeled data by a large margin. We show that emphPrompt outperforms previous approaches for in-context learning with few labeled data by a large margin.
  arXiv  Detail & Related papers  (2023-11-06T05:24:06Z)
• SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL (extended) [53.95151604061761]
  This paper introduces the framework for enhancing Text-to- filtering using large language models (LLMs) With few-shot prompting, we explore the effectiveness of consistency decoding with execution-based error analyses. With instruction fine-tuning, we delve deep in understanding the critical paradigms that influence the performance of tuned LLMs.
  arXiv  Detail & Related papers  (2023-05-26T21:39:05Z)
• UNITE: A Unified Benchmark for Text-to-SQL Evaluation [72.72040379293718]
  We introduce a UNIfied benchmark for Text-to-domain systems. It is composed of publicly available text-to-domain datasets and 29K databases. Compared to the widely used Spider benchmark, we introduce a threefold increase in SQL patterns.
  arXiv  Detail & Related papers  (2023-05-25T17:19:52Z)
• Weakly Supervised Text-to-SQL Parsing through Question Decomposition [53.22128541030441]
  We take advantage of the recently proposed question meaning representation called QDMR. Given questions, their QDMR structures (annotated by non-experts or automatically predicted) and the answers, we are able to automatically synthesizesql queries. Our results show that the weakly supervised models perform competitively with those trained on NL- benchmark data.
  arXiv  Detail & Related papers  (2021-12-12T20:02:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.