RASAT: Integrating Relational Structures into Pretrained Seq2Seq Model for Text-to-SQL

• URL: http://arxiv.org/abs/2205.06983v1
• Date: Sat, 14 May 2022 06:27:40 GMT
• Title: RASAT: Integrating Relational Structures into Pretrained Seq2Seq Model for Text-to-SQL
• Authors: Jiexing Qi, Jingyao Tang, Ziwei He, Xiangpeng Wan, Chenghu Zhou, Xinbing Wang, Quanshi Zhang, Zhouhan Lin
• Abstract summary: We propose a Transformer seq2seq architecture augmented with relationaware self-attention. Our model is able to incorporate almost all types of existing relations in the literature.
• Score: 37.173390754207766
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Relational structures such as schema linking and schema encoding have been validated as a key component to qualitatively translating natural language into SQL queries. However, introducing these structural relations comes with prices: they often result in a specialized model structure, which largely prohibits the use of large pretrained models in text-to-SQL. To address this problem, we propose RASAT: a Transformer seq2seq architecture augmented with relation-aware self-attention that could leverage a variety of relational structures while at the meantime being able to effectively inherit the pretrained parameters from the T5 model. Our model is able to incorporate almost all types of existing relations in the literature, and in addition, we propose to introduce co-reference relations for the multi-turn scenario. Experimental results on three widely used text-to-SQL datasets, covering both single-turn and multi-turn scenarios, have shown that RASAT could achieve competitive results in all three benchmarks, achieving state-of-the-art performance in execution accuracy (80.5\% EX on Spider, 53.1\% IEX on SParC, and 37.5\% IEX on CoSQL).

Related papers

• Enhancing LLM Fine-tuning for Text-to-SQLs by SQL Quality Measurement [1.392448435105643]
  Text-to-s enables non-expert users to effortlessly retrieve desired information from databases using natural language queries. Current state-of-the-art (SOTA) models like GPT4 and T5 have shown impressive performance on large-scale benchmarks like BIRD. This paper proposed a novel approach that only needs SQL Quality to enhance Text-to-s performance.
  arXiv  Detail & Related papers  (2024-10-02T17:21:51Z)
• UNITE: A Unified Benchmark for Text-to-SQL Evaluation [72.72040379293718]
  We introduce a UNIfied benchmark for Text-to-domain systems. It is composed of publicly available text-to-domain datasets and 29K databases. Compared to the widely used Spider benchmark, we introduce a threefold increase in SQL patterns.
  arXiv  Detail & Related papers  (2023-05-25T17:19:52Z)
• Conversational Text-to-SQL: An Odyssey into State-of-the-Art and Challenges Ahead [6.966624873109535]
  State-of-the-art (SOTA) systems use large, pre-trained and finetuned language models, such as the T5-family. With multi-tasking (MT) over coherent tasks with discrete prompts during training, we improve over specialized text-to-three models. We conduct studies to tease apart errors attributable to domain and compositional generalization.
  arXiv  Detail & Related papers  (2023-02-21T23:15:33Z)
• Augmenting Multi-Turn Text-to-SQL Datasets with Self-Play [46.07002748587857]
  We explore augmenting the training datasets using self-play, which leverages contextual information to synthesize new interactions. We find that self-play improves the accuracy of a strong baseline on SParC and Co, two widely used text-to-domain datasets.
  arXiv  Detail & Related papers  (2022-10-21T16:40:07Z)
• Proton: Probing Schema Linking Information from Pre-trained Language Models for Text-to-SQL Parsing [66.55478402233399]
  We propose a framework to elicit relational structures via a probing procedure based on Poincar'e distance metric. Compared with commonly-used rule-based methods for schema linking, we found that probing relations can robustly capture semantic correspondences. Our framework sets new state-of-the-art performance on three benchmarks.
  arXiv  Detail & Related papers  (2022-06-28T14:05:25Z)
• UniSAr: A Unified Structure-Aware Autoregressive Language Model for Text-to-SQL [48.21638676148253]
  We present UniSAr (Unified Structure-Aware Autoregressive Language Model), which benefits from using an off-the-shelf language model. Specifically, UniSAr extends existing autoregressive models to incorporate three non-invasive extensions to make them structure-aware.
  arXiv  Detail & Related papers  (2022-03-15T11:02:55Z)
• Learning Contextual Representations for Semantic Parsing with Generation-Augmented Pre-Training [86.91380874390778]
  We present Generation-Augmented Pre-training (GAP), that jointly learns representations of natural language utterances and table schemas by leveraging generation models to generate pre-train data. Based on experimental results, neural semantics that leverage GAP MODEL obtain new state-of-the-art results on both SPIDER and CRITERIA-TO-generative benchmarks.
  arXiv  Detail & Related papers  (2020-12-18T15:53:50Z)
• GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing [117.98107557103877]
  We present GraPPa, an effective pre-training approach for table semantic parsing. We construct synthetic question-pairs over high-free tables via a synchronous context-free grammar. To maintain the model's ability to represent real-world data, we also include masked language modeling.
  arXiv  Detail & Related papers  (2020-09-29T08:17:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.