STAR: SQL Guided Pre-Training for Context-dependent Text-to-SQL Parsing

• URL: http://arxiv.org/abs/2210.11888v1
• Date: Fri, 21 Oct 2022 11:30:07 GMT
• Title: STAR: SQL Guided Pre-Training for Context-dependent Text-to-SQL Parsing
• Authors: Zefeng Cai, Xiangyu Li, Binyuan Hui, Min Yang, Bowen Li, Binhua Li, Zheng Cao, Weijie Li, Fei Huang, Luo Si, Yongbin Li
• Abstract summary: We propose a novel pre-training framework STAR for context-dependent text-to- parsing. In addition, we construct a large-scale context-dependent text-to-the-art conversation corpus to pre-train STAR. Extensive experiments show that STAR achieves new state-of-the-art performance on two downstream benchmarks.
• Score: 64.80483736666123
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we propose a novel SQL guided pre-training framework STAR for context-dependent text-to-SQL parsing, which leverages contextual information to enrich natural language (NL) utterance and table schema representations for text-to-SQL conversations. Concretely, we propose two novel pre-training objectives which respectively explore the context-dependent interactions of NL utterances and SQL queries within each text-to-SQL conversation: (i) schema state tracking (SST) objective that tracks and explores the schema states of context-dependent SQL queries in the form of schema-states by predicting and updating the value of each schema slot during interaction; (ii) utterance dependency tracking (UDT) objective that employs weighted contrastive learning to pull together two semantically similar NL utterances and push away the representations of semantically dissimilar NL utterances within each conversation. In addition, we construct a high-quality large-scale context-dependent text-to-SQL conversation corpus to pre-train STAR. Extensive experiments show that STAR achieves new state-of-the-art performance on two downstream benchmarks (SParC and CoSQL), significantly outperforming previous pre-training methods and ranking first on the leaderboard. We believe the release of the constructed corpus, codebase and pre-trained STAR checkpoints would push forward the research in this area. For reproducibility, we release our code and data at https://github.com/AlibabaResearch/DAMO-ConvAI/tree/main/star.

Related papers

• Augmenting Multi-Turn Text-to-SQL Datasets with Self-Play [46.07002748587857]
  We explore augmenting the training datasets using self-play, which leverages contextual information to synthesize new interactions. We find that self-play improves the accuracy of a strong baseline on SParC and Co, two widely used text-to-domain datasets.
  arXiv  Detail & Related papers  (2022-10-21T16:40:07Z)
• A Survey on Text-to-SQL Parsing: Concepts, Methods, and Future Directions [102.8606542189429]
  The goal of text-to-corpora parsing is to convert a natural language (NL) question to its corresponding structured query language () based on the evidences provided by databases. Deep neural networks have significantly advanced this task by neural generation models, which automatically learn a mapping function from an input NL question to an output query.
  arXiv  Detail & Related papers  (2022-08-29T14:24:13Z)
• Proton: Probing Schema Linking Information from Pre-trained Language Models for Text-to-SQL Parsing [66.55478402233399]
  We propose a framework to elicit relational structures via a probing procedure based on Poincar'e distance metric. Compared with commonly-used rule-based methods for schema linking, we found that probing relations can robustly capture semantic correspondences. Our framework sets new state-of-the-art performance on three benchmarks.
  arXiv  Detail & Related papers  (2022-06-28T14:05:25Z)
• HIE-SQL: History Information Enhanced Network for Context-Dependent Text-to-SQL Semantic Parsing [1.343950231082215]
  We propose a History Information Enhanced text-to-the-art model (HIE-) to exploit context-dependence information from both history utterances and the last predictedsql query. We show our methods improve the performance of HIE- by a significant margin, which achieves new state-of-the-art results on the two context-dependent text-to-the-art benchmarks.
  arXiv  Detail & Related papers  (2022-03-14T11:58:37Z)
• Pay More Attention to History: A Context Modeling Strategy for Conversational Text-to-SQL [8.038535788630542]
  One of the most intractable problem of conversational text-to- domain is modeling the semantics of multi-turn queries. This paper shows that explicit modeling the semantic changes by adding each turn and the summarization of the whole context can bring better performance.
  arXiv  Detail & Related papers  (2021-12-16T09:41:04Z)
• Weakly Supervised Text-to-SQL Parsing through Question Decomposition [53.22128541030441]
  We take advantage of the recently proposed question meaning representation called QDMR. Given questions, their QDMR structures (annotated by non-experts or automatically predicted) and the answers, we are able to automatically synthesizesql queries. Our results show that the weakly supervised models perform competitively with those trained on NL- benchmark data.
  arXiv  Detail & Related papers  (2021-12-12T20:02:42Z)
• Tracking Interaction States for Multi-Turn Text-to-SQL Semantic Parsing [44.0348697408427]
  The task of multi-turn text-to- semantic parsing aims to translate natural language utterances in an interaction intosql queries. A graph relational network and a non-linear layer are designed to update the representations of these two states respectively. Experimental results on the challenging Co dataset demonstrate the effectiveness of our proposed method.
  arXiv  Detail & Related papers  (2020-12-09T11:59:58Z)
• IGSQL: Database Schema Interaction Graph Based Neural Model for Context-Dependent Text-to-SQL Generation [61.09660709356527]
  We propose a database schema interaction graph encoder to utilize historicalal information of database schema items. We evaluate our model on the benchmark SParC and Co datasets.
  arXiv  Detail & Related papers  (2020-11-11T12:56:21Z)
• GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing [117.98107557103877]
  We present GraPPa, an effective pre-training approach for table semantic parsing. We construct synthetic question-pairs over high-free tables via a synchronous context-free grammar. To maintain the model's ability to represent real-world data, we also include masked language modeling.
  arXiv  Detail & Related papers  (2020-09-29T08:17:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.