Optimizing Small Language Models for NL2SQL via Chain-of-Thought Fine-Tuning

• URL: http://arxiv.org/abs/2603.22942v1
• Date: Tue, 24 Mar 2026 08:39:06 GMT
• Title: Optimizing Small Language Models for NL2SQL via Chain-of-Thought Fine-Tuning
• Abstract summary: This paper explores the efficacy of fine-tuning both large and small language models on NL2 tasks.<n>Our research reveals a counter-intuitive scaling phenomenon.<n>Fine-tuning improved the small model baseline from 36% to 45%, and further enriching the dataset with explicit Chain-of-Thought reasoning surged accuracy to 54.5%.
• Score: 0.2519906683279153
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Translating Natural Language to SQL (NL2SQL) remains a critical bottleneck for democratization of data in enterprises. Although Large Language Models (LLMs) like Gemini 2.5 and other LLMs have demonstrated impressive zero-shot capabilities, their high inference costs limit deployment at scale. This paper explores the efficacy of fine-tuning both large and small language models on NL2SQL tasks. Our research reveals a counter-intuitive scaling phenomenon. Fine-tuning large models (Gemini 2.5 Flash/Lite) on standard datasets yields negligible returns, often leading to overfitting on complex queries. Conversely, small models (Qwen) show significant gains. Fine-tuning improved the small model baseline from 36% to 45%, and further enriching the dataset with explicit Chain-of-Thought (CoT) reasoning surged accuracy to 54.5%(Fig 2). While this is still lower than the accuracy of large models like Gemini 2.5 , it does serve the business goal of significant cost reduction, latency in inference time and also meeting the business critical performance accuracy threshold.This paper demonstrates that transferring reasoning patterns enables compute-efficient smaller models to approach production-grade performance.

Related papers

• Improving the Robustness of Large Language Models for Code Tasks via Fine-tuning with Perturbed Data [10.698357983420928]
  This work aims to improve the robustness of Large Language Models against potential adversarial inputs.<n>We systematically evaluated robustness by fine-tuning models using datasets perturbed at character-level, word-level, and sentence-level.<n>Fine-tuning models with perturbed datasets significantly improves model robustness (RD usually drops around 4% - 6%), especially for models with relatively weak robustness.
  arXiv  Detail & Related papers  (2026-02-11T22:30:01Z)
• Robust LLM-based Column Type Annotation via Prompt Augmentation with LoRA Tuning [0.749377967268953]
  Column Type.<n>CTA is a fundamental step towards enabling schema alignment and semantic understanding.<n>Existing encoder-only language models achieve high accuracy when fine-tuned on labeled columns.<n>We present a parameter-efficient framework for CTA that trains models over prompt-augmented data.
  arXiv  Detail & Related papers  (2025-12-28T02:04:17Z)
• Information Capacity: Evaluating the Efficiency of Large Language Models via Text Compression [53.39128997308138]
  We introduce information capacity, a measure of model efficiency based on text compression performance.<n> Empirical evaluations on mainstream open-source models show that models of varying sizes within a series exhibit consistent information capacity.<n>A distinctive feature of information capacity is that it incorporates tokenizer efficiency, which affects both input and output token counts.
  arXiv  Detail & Related papers  (2025-11-11T10:07:32Z)
• Breaking the Myth: Can Small Models Infer Postconditions Too? [15.725275719200303]
  We show that a small, fine-tuned language model can achieve high-quality postcondition generation with much lower computational costs.<n>Our approach tackles real-world repository dependencies and preserves pre-state information, allowing for expressive and accurate specifications.
  arXiv  Detail & Related papers  (2025-07-14T11:44:04Z)
• EpiCoDe: Boosting Model Performance Beyond Training with Extrapolation and Contrastive Decoding [50.29046178980637]
  EpiCoDe is a method that boosts model performance in data-scarcity scenarios without extra training.<n>We show that EpiCoDe consistently outperforms existing methods with significant and robust improvement.
  arXiv  Detail & Related papers  (2025-06-04T02:11:54Z)
• InkubaLM: A small language model for low-resource African languages [9.426968756845389]
  InkubaLM is a small language model with 0.4 billion parameters. It achieves performance comparable to models with significantly larger parameter counts. It demonstrates remarkable consistency across multiple languages.
  arXiv  Detail & Related papers  (2024-08-30T05:42:31Z)
• Improving Large Models with Small models: Lower Costs and Better Performance [81.55672406002715]
  We propose Data Shunt$+$ (DS$+$), a general paradigm for collaboration of small and large models. For instance, ChatGPT achieves an accuracy of $94.43%$ on Amazon Product sentiment analysis, and DS$+$ achieves an accuracy of $95.64%$, while the cost has been reduced to only $31.18%$.
  arXiv  Detail & Related papers  (2024-06-15T14:44:43Z)
• MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies [85.57899012821211]
  Small Language Models (SLMs) are a resource-efficient alternative to Large Language Models (LLMs) We introduce MiniCPM, specifically the 1.2B and 2.4B non-embedding parameter variants. We also introduce MiniCPM family, including MiniCPM-DPO, MiniCPM-MoE and MiniCPM-128K.
  arXiv  Detail & Related papers  (2024-04-09T15:36:50Z)
• DTS-SQL: Decomposed Text-to-SQL with Small Large Language Models [7.388002745070808]
  We introduce a novel two-stage fine-tuning approach that decomposes the task into two simpler tasks. We show that this approach improves execution accuracy by 3 to 7 percent.
  arXiv  Detail & Related papers  (2024-02-02T03:21:00Z)
• Scaling Relationship on Learning Mathematical Reasoning with Large Language Models [75.29595679428105]
  We investigate how the pre-training loss, supervised data amount, and augmented data amount influence the reasoning performances of a supervised LLM. We find that rejection samples from multiple models push LLaMA-7B to an accuracy of 49.3% on GSM8K which outperforms the supervised fine-tuning (SFT) accuracy of 35.9% significantly.
  arXiv  Detail & Related papers  (2023-08-03T15:34:01Z)
• Speculative Decoding with Big Little Decoder [108.95187338417541]
  Big Little Decoder (BiLD) is a framework that can improve inference efficiency and latency for a wide range of text generation applications. On an NVIDIA T4 GPU, our framework achieves a speedup of up to 2.12x speedup with minimal generation quality degradation. Our framework is fully plug-and-play and can be applied without any modifications in the training process or model architecture.
  arXiv  Detail & Related papers  (2023-02-15T18:55:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.