CAPER: Clause-Aligned Process Supervision for Text-to-SQL

• URL: http://arxiv.org/abs/2606.03327v1
• Date: Tue, 02 Jun 2026 08:35:40 GMT
• Title: CAPER: Clause-Aligned Process Supervision for Text-to-SQL
• Abstract summary: We propose CAPER, which automatically derives clause-level supervision via counterfactual intervention on the SQL abstract syntax tree.<n>The resulting data is used to train CAPER-9B, a lightweight Clause-boundary feedback for policy optimization and candidate verification.<n> Experiments on BIRD and Spider show that clause-aligned supervision not only improves execution accuracy, achieving up to a 15.3% relative EX improvement over GPT-5.4, but also strengthens failure-localization capability, reaching 84.53% accuracy and 90.60% MRR on held-out failures.
• Score: 11.286563956844946
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Text-to-SQL systems are typically evaluated by query-level execution correctness, but this terminal signal provides little guidance about which intermediate SQL decision caused success or failure. Token-level dense supervision is also ill-suited: SQL tokens do not align with complete semantic decisions, can penalize execution-equivalent queries, and are difficult to label reliably at scale. We therefore propose CAPER, which automatically derives clause-level supervision via counterfactual intervention on the SQL abstract syntax tree, enabling root-cause error localization for reward modeling; the resulting data is used to train CAPER-9B, a lightweight Clause-PRM that provides clause-boundary feedback for policy optimization and candidate verification. Experiments on BIRD and Spider show that clause-aligned supervision not only improves execution accuracy, achieving up to a 15.3% relative EX improvement over GPT-5.4, but also strengthens failure-localization capability, reaching 84.53% accuracy and 90.60% MRR on held-out failures. Our project page is at https://github.com/banrichard/RL-NL2SQL.

Related papers

• ACTS-SQL: Agentic and Critic-Oriented Tree-Structured SQL Correctness with Large Language Models [27.75092433893197]
  We present a training-free framework that formulatessql correction as a plan-guided, tree-structured debug process.<n>We observe consistent accuracy gains over strong LLM backbones and representative agent-based baselines, achieving a 9.42% improvement over the previous state-of-the-art method.<n>The framework is also deployed in the Torch Log Service (TLS) of Volcano Engine to support an online Text-to-TLS API.
  arXiv  Detail & Related papers  (2026-08-15T09:43:57Z)
• SERL-SQL: Selective Hindsight Distillation for Text-to-SQL Reinforcement Agentic Learning [11.822926017805024]
  SERL- is a selective execution-grounded reinforcement learning framework for multi-turn Text-to- agents.<n>Uses a training-only teacher to re-score student actions with execution feedback.<n>Experiments show that SERL- achieves competitive performance, reaching 76.56% execution accuracy on Spider-Test.
  arXiv  Detail & Related papers  (2026-08-01T07:24:12Z)
• Spider 2.0-AIFunc: Extending Real-World Text-to-SQL to AI-Native SQL Workflows [54.49251107006261]
  We introduce Spider 2.0-AIFunc, a benchmark of 465 verified instances across 125 real-world databases covering six types of AI functions on the Snowflake platform.<n>We find that the strongest proprietary models reach 67-70% execution accuracy while the best open-source model achieves 58.1%.
  arXiv  Detail & Related papers  (2026-07-07T12:54:08Z)
• EXPO-SQL: Execution-based Clause-level Policy Optimization for Text-to-SQL [30.5119607116771]
  Text-to- querying enables users to query databases using natural language.<n>Existing Large Language Models based reinforcement learning methods treat correct and incorrect clauses equally.<n>We propose EXPO- (EXecution-based clause-level Policy Optimization for Text-to- querying) which provides fine-grained supervision through clause-level rewards.
  arXiv  Detail & Related papers  (2026-04-29T10:33:16Z)
• PV-SQL: Synergizing Database Probing and Rule-based Verification for Text-to-SQL Agents [7.865833770467325]
  Text-to- systems often struggle with deep contextual understanding, particularly for complex queries with subtle requirements.<n>We present PV-Verify, an agentic framework that addresses these failures through two complementary components: Probe and Verify.<n>The Probe component iteratively generates probing queries to retrieve concrete records from the database, resolving ambiguities in value formats, column semantics, and inter-table relationships to build richer contextual understanding.
  arXiv  Detail & Related papers  (2026-04-19T22:54:05Z)
• ErrorLLM: Modeling SQL Errors for Text-to-SQL Refinement [57.98138819417949]
  We propose ErrorLLM, a framework that explicitly models text-to- querying.<n>We show that ErrorLLM achieves the most significant improvements over backbone initial generation.<n>ErrorLLM addresses both sides by high detection F1 score while maintaining refinement effectiveness.
  arXiv  Detail & Related papers  (2026-03-04T05:27:20Z)
• Bridging Global Intent with Local Details: A Hierarchical Representation Approach for Semantic Validation in Text-to-SQL [30.78817492504152]
  HERO is a hierarchical representation approach that integrates global intent and local details.<n>We employ a Nested Message Passing Neural Network (NMPNN) to capture inherent information in relational schema-guided semantics.<n>Our approach outperforms existing state-of-the-art methods, achieving an average 9.40% improvement of AUPRC and 12.35% of AUROC in identifying semantic inconsistencies.<n>It excels at detecting fine-grained semantic errors, provides large language models with more granular feedback, and ultimately enhances the reliability and interpretability of data querying platforms.
  arXiv  Detail & Related papers  (2025-12-28T02:25:33Z)
• Text-to-SQL as Dual-State Reasoning: Integrating Adaptive Context and Progressive Generation [54.53145282349042]
  We introduce DSR-sourced, a textbfDual-textbfS textbfReasoning framework that models Text-to-context as an interaction between an adaptive context state and a progressive generation state.<n>Without any post-training or in-context examples, DSR-sourced achieves competitive performance, reaching 35.28% execution accuracy on Spider 2.0-Snow and 68.32% on BIRD development set.
  arXiv  Detail & Related papers  (2025-11-26T13:52:50Z)
• GBV-SQL: Guided Generation and SQL2Text Back-Translation Validation for Multi-Agent Text2SQL [12.455525963127497]
  GBV- is a novel multi-agent framework that introduces Guided Generation with SQL2Text Back-translation Validation.<n>This mechanism uses a specialized agent to translate the generatedsql back into natural language, which verifies its logical alignment with the original question.<n>We introduce a formal typology for "Gold Errors", which are pervasive flaws in the ground-truth and demonstrate how they obscure true model performance.
  arXiv  Detail & Related papers  (2025-09-16T03:21:12Z)
• SQLens: An End-to-End Framework for Error Detection and Correction in Text-to-SQL [20.93676525997898]
  We propose an end-to-end framework for fine-grained detection and correction of semantic errors in large language models (LLMs) generated by text-to-the-box systems.<n>We show that our framework outperforms the best LLM-based self-evaluation method by 25.78% in F1 for error detection, and improves execution accuracy of out-of-the-box systems by up to 20%.
  arXiv  Detail & Related papers  (2025-06-04T22:25:47Z)
• RSL-SQL: Robust Schema Linking in Text-to-SQL Generation [51.00761167842468]
  We propose a novel framework called RSL- that combines bidirectional schema linking, contextual information augmentation, binary selection strategy, and multi-turn self-correction. benchmarks demonstrate that our approach achieves SOTA execution accuracy among open-source solutions, with 67.2% on BIRD and 87.9% on GPT-4ocorrection. Our approach outperforms a series of GPT-4 based Text-to-Seek systems when adopting DeepSeek (much cheaper) with same intact prompts.
  arXiv  Detail & Related papers  (2024-10-31T16:22:26Z)
• SelECT-SQL: Self-correcting ensemble Chain-of-Thought for Text-to-SQL [3.422309388045878]
  We introduce SelECT-, a novel in-context learning solution that uses an algorithmic combination of chain-of-thought, self-correction, and ensemble methods. Specifically, when configured using GPT as the base LLM, SelECT-Turbo achieves 84.2% execution accuracy on the Spider leaderboard's development set.
  arXiv  Detail & Related papers  (2024-09-16T05:40:18Z)
• DAC: Decomposed Automation Correction for Text-to-SQL [51.48239006107272]
  We introduce De Automation Correction (DAC), which corrects text-to-composed by decomposing entity linking and skeleton parsing. We show that our method improves performance by $3.7%$ on average of Spider, Bird, and KaggleDBQA compared with the baseline method.
  arXiv  Detail & Related papers  (2024-08-16T14:43:15Z)
• Weakly Supervised Text-to-SQL Parsing through Question Decomposition [53.22128541030441]
  We take advantage of the recently proposed question meaning representation called QDMR. Given questions, their QDMR structures (annotated by non-experts or automatically predicted) and the answers, we are able to automatically synthesizesql queries. Our results show that the weakly supervised models perform competitively with those trained on NL- benchmark data.
  arXiv  Detail & Related papers  (2021-12-12T20:02:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.