Knowledge Distillation for Low-Resource Open-source Text-to-SQL Model

• URL: http://arxiv.org/abs/2605.22843v1
• Date: Wed, 13 May 2026 07:54:13 GMT
• Title: Knowledge Distillation for Low-Resource Open-source Text-to-SQL Model
• Abstract summary: We propose a knowledge-aware Text-to-tt framework that constructs task-specific knowledge base and injects it into both training and inference.<n>This framework generates contextually grounded synthetic training data and enhances inference through targeted knowledge retrieval.
• Score: 4.098313540411055
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Text-to-SQL converts natural language questions into executable SQL queries, enabling non-technical users to access relational databases for analytics and intelligent data services. In real-world scenarios, performance is often constrained by low-resource settings, where high-quality annotated \texttt{<question, SQL>} pairs are scarce, particularly for domain-specific databases. Additional challenges include opaque schema definitions, abbreviations, and implicit business logic that are not explicitly encoded in the schema. Existing data synthesis and prompting techniques improve coverage but often fail to produce task-specific, semantically grounded examples aligned with database constraints. To address these challenges, we propose a knowledge-aware Text-to-SQL framework that constructs task-specific knowledge base including schema semantics, abbreviations, business logic, and query patterns, and injects them into both training and inference. This framework generates diverse, contextually grounded synthetic training data and enhances inference through targeted knowledge retrieval. Experiments on seven benchmarks, covering both general and domain-specific datasets, demonstrate that our approach substantially improves the performance of open-source and closed-source large language models in Text-to-SQL tasks, especially in low-resource domain-specific settings, enhancing generalization, robustness, and adaptability.

Related papers

• Skeletons Matter: Dynamic Data Augmentation for Text-to-Query [66.52311036179294]
  We formally define the Text-to-Query task paradigm, unifying semantic parsing tasks across various query languages.<n>We identify query skeletons as a shared optimization target of Text-to-Query tasks, and propose a general dynamic data augmentation framework.<n> Experiments on four Text-to-Query benchmarks demonstrate that our method achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-11-24T09:39:03Z)
• ORANGE: An Online Reflection ANd GEneration framework with Domain Knowledge for Text-to-SQL [8.241433772695018]
  Large Language Models (LLMs) have demonstrated remarkable progress in translating natural language tosql.<n>A significant semantic gap persists between their general knowledge and domain-specific semantics of databases.<n>We introduce Orange, an online self-evolutionary framework that constructs database-specific knowledge bases by parsing queries from translation logs.
  arXiv  Detail & Related papers  (2025-11-02T15:57:18Z)
• Who Gets Cited Most? Benchmarking Long-Context Language Models on Scientific Articles [81.89404347890662]
  SciTrek is a novel question-answering benchmark designed to evaluate the long-context reasoning capabilities of large language models (LLMs) using scientific articles.<n>Our analysis reveals systematic shortcomings in models' abilities to perform basic numerical operations and accurately locate specific information in long contexts.
  arXiv  Detail & Related papers  (2025-09-25T11:36:09Z)
• Data Dependency-Aware Code Generation from Enhanced UML Sequence Diagrams [54.528185120850274]
  We propose a novel step-by-step code generation framework named API2Dep.<n>First, we introduce an enhanced Unified Modeling Language (UML) API diagram tailored for service-oriented architectures.<n>Second, recognizing the critical role of data flow, we introduce a dedicated data dependency inference task.
  arXiv  Detail & Related papers  (2025-08-05T12:28:23Z)
• RASL: Retrieval Augmented Schema Linking for Massive Database Text-to-SQL [1.3654846342364308]
  We introduce a component-based retrieval architecture that decomposes database schemas and metadata into discrete semantic units.<n>Our solution enables practical text-to- interfaces across diverse enterprise settings without specialized fine-tuning.
  arXiv  Detail & Related papers  (2025-07-30T21:09:47Z)
• Knowledge Base Construction for Knowledge-Augmented Text-to-SQL [37.87911346522774]
  We propose constructing a knowledge base for text-to-one, a foundational source of knowledge, from which we generate necessary knowledge for given queries.<n>Our knowledge base is comprehensive, which is constructed based on a combination of all available questions and their associated database schemas.<n>We validate our approach on multiple text-to-one datasets, considering both overlapping and non-overlapping database scenarios.
  arXiv  Detail & Related papers  (2025-05-28T08:17:58Z)
• Text2Schema: Filling the Gap in Designing Database Table Structures based on Natural Language [22.15408079332362]
  People without a database background usually rely on file systems or tools such as Excel data management.<n> Database systems possess strong management capabilities, but require a high level of professional expertise from users.
  arXiv  Detail & Related papers  (2025-03-31T09:39:19Z)
• Bridging the Gap: Enabling Natural Language Queries for NoSQL Databases through Text-to-NoSQL Translation [25.638927795540454]
  We introduce the Text-to-No task, which aims to convert natural language queries into accessible queries.<n>To promote research in this area, we released a large-scale and open-source dataset for this task, named TEND (short interfaces for Text-to-No dataset)<n>We also designed a SLM (Small Language Model)-assisted and RAG (Retrieval-augmented Generation)-assisted multi-step framework called SMART, which is specifically designed for Text-to-No conversion.
  arXiv  Detail & Related papers  (2025-02-16T17:01:48Z)
• AMBROSIA: A Benchmark for Parsing Ambiguous Questions into Database Queries [56.82807063333088]
  We introduce a new benchmark, AMBROSIA, which we hope will inform and inspire the development of text-to-open programs. Our dataset contains questions showcasing three different types of ambiguity (scope ambiguity, attachment ambiguity, and vagueness) In each case, the ambiguity persists even when the database context is provided. This is achieved through a novel approach that involves controlled generation of databases from scratch.
  arXiv  Detail & Related papers  (2024-06-27T10:43:04Z)
• DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text [73.68051228972024]
  Large Language Models (LLMs) have exhibited impressive generation capabilities, but they suffer from hallucinations when relying on their internal knowledge. Retrieval-augmented LLMs have emerged as a potential solution to ground LLMs in external knowledge.
  arXiv  Detail & Related papers  (2023-10-31T04:37:57Z)
• SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL (extended) [53.95151604061761]
  This paper introduces the framework for enhancing Text-to- filtering using large language models (LLMs) With few-shot prompting, we explore the effectiveness of consistency decoding with execution-based error analyses. With instruction fine-tuning, we delve deep in understanding the critical paradigms that influence the performance of tuned LLMs.
  arXiv  Detail & Related papers  (2023-05-26T21:39:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.