SERL-SQL: Selective Hindsight Distillation for Text-to-SQL Reinforcement Agentic Learning

• URL: http://arxiv.org/abs/2608.00485v2
• Date: Tue, 04 Aug 2026 08:32:19 GMT
• Title: SERL-SQL: Selective Hindsight Distillation for Text-to-SQL Reinforcement Agentic Learning
• Abstract summary: SERL- is a selective execution-grounded reinforcement learning framework for multi-turn Text-to- agents.<n>Uses a training-only teacher to re-score student actions with execution feedback.<n>Experiments show that SERL- achieves competitive performance, reaching 76.56% execution accuracy on Spider-Test.
• Score: 11.822926017805024
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent Text-to-SQL systems increasingly rely on multi-turn interaction, execution feedback, and reinforcement learning. However, most existing methods use execution correctness only as a trajectory-level reward, which provides limited guidance for identifying the SQL decisions responsible for success or failure. We propose SERL-SQL, a selective execution-grounded reinforcement learning framework for multi-turn Text-to-SQL agents. SERL-SQL samples on-policy SQL interaction trajectories and uses a training-only teacher to re-score student actions with execution feedback. The resulting teacher--student likelihood gap is converted into bounded, masked weights that reweight GRPO advantages only on SQL and tool-action tokens. In this way, task rewards preserve the optimization direction, while execution hindsight provides localized credit assignment. Experiments on BIRD, Spider, and cross-domain benchmarks show that SERL-SQL achieves competitive performance, reaching 76.56% execution accuracy on BIRD-Dev and 89.92% on Spider-Test. Moreover, our reward-based selection strategy closely approaches the oracle Best-of-N upper bound and consistently outperforms consistency-based selection, showing that SERL-SQL produces high-quality candidates that can be reliably identified by lightweight execution-grounded rewards. Our code will be released at https://github.com/Ffunkytao/SERL-SQL.

Related papers

• FINER-SQL: Boosting Small Language Models for Text-to-SQL [40.19592881059662]
  Large language models suffer from high computational cost, long latency, and data privacy concerns.<n>A natural alternative is to use small language models, which enable efficient and private on-premise deployment.<n>We propose FINER-sql, a scalable and reusable reinforcement learning framework.
  arXiv  Detail & Related papers  (2026-05-05T07:51:29Z)
• EXPO-SQL: Execution-based Clause-level Policy Optimization for Text-to-SQL [30.5119607116771]
  Text-to- querying enables users to query databases using natural language.<n>Existing Large Language Models based reinforcement learning methods treat correct and incorrect clauses equally.<n>We propose EXPO- (EXecution-based clause-level Policy Optimization for Text-to- querying) which provides fine-grained supervision through clause-level rewards.
  arXiv  Detail & Related papers  (2026-04-29T10:33:16Z)
• R$^3$-SQL: Ranking Reward and Resampling for Text-to-SQL [55.967050404665606]
  R$3$- achieves 75.03 execution accuracy on BIRD-dev, a new state of the art among methods using models with disclosed sizes, with consistent gains across five benchmarks.
  arXiv  Detail & Related papers  (2026-04-28T07:40:50Z)
• Text-to-SQL as Dual-State Reasoning: Integrating Adaptive Context and Progressive Generation [54.53145282349042]
  We introduce DSR-sourced, a textbfDual-textbfS textbfReasoning framework that models Text-to-context as an interaction between an adaptive context state and a progressive generation state.<n>Without any post-training or in-context examples, DSR-sourced achieves competitive performance, reaching 35.28% execution accuracy on Spider 2.0-Snow and 68.32% on BIRD development set.
  arXiv  Detail & Related papers  (2025-11-26T13:52:50Z)
• MARS-SQL: A multi-agent reinforcement learning framework for Text-to-SQL [22.59453421744114]
  We introduce MARS-, a novel multi-agent framework that combines principled task decomposition and interactive reinforcement learning (RL)<n>Experiments show that MARS- achieves state-of-the-art Execution Accuracy of 77.84% on the BIRD set and 89.84% on the Spider test set.
  arXiv  Detail & Related papers  (2025-11-02T16:55:30Z)
• Arctic-Text2SQL-R1: Simple Rewards, Strong Reasoning in Text-to-SQL [35.21185734929167]
  We present Arctic-Text2-R1, a reinforcement learning (RL) framework and model family designed to generate accurate, executablesql.<n>Our approach avoids curated intermediate supervision and complex reward shaping, promoting stable training and alignment with the end task.<n> Notably, our 7B model outperforms prior 70B-class systems, highlighting the framework's scalability and efficiency.
  arXiv  Detail & Related papers  (2025-05-22T23:33:47Z)
• Graph-Reward-SQL: Execution-Free Reinforcement Learning for Text-to-SQL via Graph Matching and Stepwise Reward [16.47355258999193]
  Reinforcement learning (RL) has been widely adopted to enhance the performance of large language models (LLMs) on Text-to- tasks.<n>Existing methods often rely on execution-based or LLM-based Bradley-Terry reward models.<n>We propose a novel reward model framework for RL-based Text-to-score named Graph-Reward-RTM.
  arXiv  Detail & Related papers  (2025-05-18T11:53:01Z)
• RSL-SQL: Robust Schema Linking in Text-to-SQL Generation [51.00761167842468]
  We propose a novel framework called RSL- that combines bidirectional schema linking, contextual information augmentation, binary selection strategy, and multi-turn self-correction. benchmarks demonstrate that our approach achieves SOTA execution accuracy among open-source solutions, with 67.2% on BIRD and 87.9% on GPT-4ocorrection. Our approach outperforms a series of GPT-4 based Text-to-Seek systems when adopting DeepSeek (much cheaper) with same intact prompts.
  arXiv  Detail & Related papers  (2024-10-31T16:22:26Z)
• RB-SQL: A Retrieval-based LLM Framework for Text-to-SQL [48.516004807486745]
  Large language models (LLMs) with in-context learning have significantly improved the performance of text-to- task. We propose RB-, a novel retrieval-based framework for in-context prompt engineering. Experiment results demonstrate that our model achieves better performance than several competitive baselines on public datasets BIRD and Spider.
  arXiv  Detail & Related papers  (2024-07-11T08:19:58Z)
• Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation [76.76046657162306]
  Large language models (LLMs) have emerged as a new paradigm for Text-to- task. Large language models (LLMs) have emerged as a new paradigm for Text-to- task.
  arXiv  Detail & Related papers  (2023-08-29T14:59:54Z)
• SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL (extended) [53.95151604061761]
  This paper introduces the framework for enhancing Text-to- filtering using large language models (LLMs) With few-shot prompting, we explore the effectiveness of consistency decoding with execution-based error analyses. With instruction fine-tuning, we delve deep in understanding the critical paradigms that influence the performance of tuned LLMs.
  arXiv  Detail & Related papers  (2023-05-26T21:39:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.