MAG-SQL: Multi-Agent Generative Approach with Soft Schema Linking and Iterative Sub-SQL Refinement for Text-to-SQL

• URL: http://arxiv.org/abs/2408.07930v4
• Date: Thu, 7 Nov 2024 03:37:51 GMT
• Title: MAG-SQL: Multi-Agent Generative Approach with Soft Schema Linking and Iterative Sub-SQL Refinement for Text-to-SQL
• Authors: Wenxuan Xie, Gaochen Wu, Bowen Zhou,
• Abstract summary: Recent In-Context Learning based methods have achieved remarkable success in Text-to-Context task. There is still a large gap between the performance of these models and human performance on datasets with complex database schema and difficult questions, such as. In our framework, an entity-based method with tables' summary is used to select the columns in database, and a novel targets-conditions decomposition method is introduced to decompose those complex questions.
• Score: 15.824894030016187
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent In-Context Learning based methods have achieved remarkable success in Text-to-SQL task. However, there is still a large gap between the performance of these models and human performance on datasets with complex database schema and difficult questions, such as BIRD. Besides, existing work has neglected to supervise intermediate steps when solving questions iteratively with question decomposition methods, and the schema linking methods used in these works are very rudimentary. To address these issues, we propose MAG-SQL, a multi-agent generative approach with soft schema linking and iterative Sub-SQL refinement. In our framework, an entity-based method with tables' summary is used to select the columns in database, and a novel targets-conditions decomposition method is introduced to decompose those complex questions. Additionally, we build a iterative generating module which includes a Sub-SQL Generator and Sub-SQL Refiner, introducing external oversight for each step of generation. Through a series of ablation studies, the effectiveness of each agent in our framework has been demonstrated. When evaluated on the BIRD benchmark with GPT-4, MAG-SQL achieves an execution accuracy of 61.08%, compared to the baseline accuracy of 46.35% for vanilla GPT-4 and the baseline accuracy of 57.56% for MAC-SQL. Besides, our approach makes similar progress on Spider. The codes are available at https://github.com/LancelotXWX/MAG-SQL.

Related papers

• RSL-SQL: Robust Schema Linking in Text-to-SQL Generation [12.765849111313614]
  We propose a novel framework called RSL- that combines bidirectional schema linking, contextual information augmentation, binary selection strategy, and multi-turn self-correction. Experiments on the BIRD and Spider benchmarks demonstrate that our approach achieves state-of-the-art execution accuracy among open-source solutions.
  arXiv  Detail & Related papers  (2024-10-31T16:22:26Z)
• DAC: Decomposed Automation Correction for Text-to-SQL [51.48239006107272]
  We introduce De Automation Correction (DAC), which corrects text-to-composed by decomposing entity linking and skeleton parsing. We show that our method improves performance by $3.7%$ on average of Spider, Bird, and KaggleDBQA compared with the baseline method.
  arXiv  Detail & Related papers  (2024-08-16T14:43:15Z)
• CHESS: Contextual Harnessing for Efficient SQL Synthesis [1.9506402593665235]
  We propose a new pipeline that retrieves relevant data and context, selects an efficient schema, and synthesizes correct and efficient queries. Our method achieves new state-of-the-art performance on the cross-domain challenging BIRD dataset.
  arXiv  Detail & Related papers  (2024-05-27T01:54:16Z)
• MCS-SQL: Leveraging Multiple Prompts and Multiple-Choice Selection For Text-to-SQL Generation [10.726734105960924]
  Large language models (LLMs) have enabled in-context learning (ICL)-based methods that significantly outperform fine-tuning approaches for text-to- tasks. This study considers the sensitivity of LLMs to the prompts and introduces a novel approach that leverages multiple prompts to explore a broader search space for possible answers. We establish a new SOTA performance on the BIRD in terms of both the accuracy and efficiency of the generated queries.
  arXiv  Detail & Related papers  (2024-05-13T04:59:32Z)
• Schema-Aware Multi-Task Learning for Complex Text-to-SQL [4.913409359995421]
  We present a schema-aware multi-task learning framework (named MT) for complicatedsql queries. Specifically, we design a schema linking discriminator module to distinguish the valid question-schema linkings. On the decoder side, we define 6-type relationships to describe the connections between tables and columns.
  arXiv  Detail & Related papers  (2024-03-09T01:13:37Z)
• MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQL [47.120862170230566]
  Recent Text-to-yourself methods usually suffer from significant performance degradation on "huge" databases. We introduce MAC, a novel Text-to-yourself LLM-based multi-agent collaborative framework. In our framework, we leverage GPT-4 as the strong backbone for all agent tasks to determine the upper bound of our framework. We then fine-tune an open-sourced instruction-followed model,sql-Llama, by leveraging Code 7B, to accomplish all tasks as GPT-4 does.
  arXiv  Detail & Related papers  (2023-12-18T14:40:20Z)
• SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL (extended) [53.95151604061761]
  This paper introduces the framework for enhancing Text-to- filtering using large language models (LLMs) With few-shot prompting, we explore the effectiveness of consistency decoding with execution-based error analyses. With instruction fine-tuning, we delve deep in understanding the critical paradigms that influence the performance of tuned LLMs.
  arXiv  Detail & Related papers  (2023-05-26T21:39:05Z)
• Proton: Probing Schema Linking Information from Pre-trained Language Models for Text-to-SQL Parsing [66.55478402233399]
  We propose a framework to elicit relational structures via a probing procedure based on Poincar'e distance metric. Compared with commonly-used rule-based methods for schema linking, we found that probing relations can robustly capture semantic correspondences. Our framework sets new state-of-the-art performance on three benchmarks.
  arXiv  Detail & Related papers  (2022-06-28T14:05:25Z)
• S$^2$SQL: Injecting Syntax to Question-Schema Interaction Graph Encoder for Text-to-SQL Parsers [66.78665327694625]
  We propose S$2$, injecting Syntax to question- encoder graph for Text-to- relational parsing. We also employ the decoupling constraint to induce diverse edge embedding, which further improves the network's performance. Experiments on the Spider and robustness setting Spider-Syn demonstrate that the proposed approach outperforms all existing methods when pre-training models are used.
  arXiv  Detail & Related papers  (2022-03-14T09:49:15Z)
• Bridging Textual and Tabular Data for Cross-Domain Text-to-SQL Semantic Parsing [110.97778888305506]
  BRIDGE represents the question and DB schema in a tagged sequence where a subset of the fields are augmented with cell values mentioned in the question. BRIDGE attained state-of-the-art performance on popular cross-DB text-to- relational benchmarks. Our analysis shows that BRIDGE effectively captures the desired cross-modal dependencies and has the potential to generalize to more text-DB related tasks.
  arXiv  Detail & Related papers  (2020-12-23T12:33:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.