Prefix-to-SQL: Text-to-SQL Generation from Incomplete User Questions

• URL: http://arxiv.org/abs/2109.13066v3
• Date: Thu, 30 Sep 2021 02:45:10 GMT
• Title: Prefix-to-SQL: Text-to-SQL Generation from Incomplete User Questions
• Authors: Naihao Deng, Shuaichen Chang, Peng Shi, Tao Yu, Rui Zhang
• Abstract summary: We propose a new task, prefix-to-Query, which takes question prefix from users as the input and predicts the intendedsql. We construct a new benchmark called PAGSAS that contains 124K user question prefixes and the intendedsql for 5 sub-tasks Advising, GeoQuery, Scholar, ATIS, and Spider. As we observe the difficulty of prefix-to-Query is related to the number of omitted tokens, we incorporate curriculum learning of feeding examples with an increasing number of omitted tokens.
• Score: 33.48258057604425
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Existing text-to-SQL research only considers complete questions as the input, but lay-users might strive to formulate a complete question. To build a smarter natural language interface to database systems (NLIDB) that also processes incomplete questions, we propose a new task, prefix-to-SQL which takes question prefix from users as the input and predicts the intended SQL. We construct a new benchmark called PAGSAS that contains 124K user question prefixes and the intended SQL for 5 sub-tasks Advising, GeoQuery, Scholar, ATIS, and Spider. Additionally, we propose a new metric SAVE to measure how much effort can be saved by users. Experimental results show that PAGSAS is challenging even for strong baseline models such as T5. As we observe the difficulty of prefix-to-SQL is related to the number of omitted tokens, we incorporate curriculum learning of feeding examples with an increasing number of omitted tokens. This improves scores on various sub-tasks by as much as 9% recall scores on sub-task GeoQuery in PAGSAS.

Related papers

• Decoupling SQL Query Hardness Parsing for Text-to-SQL [2.30258928355895]
  We introduce an innovative framework for Text-to-coupled based on decoupling query hardness parsing. This framework decouples the Text-to-couple task based on query hardness by analyzing questions and schemas, simplifying the multi-hardness task into a single-hardness challenge.
  arXiv  Detail & Related papers  (2023-12-11T07:20:46Z)
• UNITE: A Unified Benchmark for Text-to-SQL Evaluation [72.72040379293718]
  We introduce a UNIfied benchmark for Text-to-domain systems. It is composed of publicly available text-to-domain datasets and 29K databases. Compared to the widely used Spider benchmark, we introduce a threefold increase in SQL patterns.
  arXiv  Detail & Related papers  (2023-05-25T17:19:52Z)
• Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs [89.68522473384522]
  We present Bird, a big benchmark for large-scale database grounded in text-to-efficient tasks. Our emphasis on database values highlights the new challenges of dirty database contents. Even the most effective text-to-efficient models, i.e. ChatGPT, achieves only 40.08% in execution accuracy.
  arXiv  Detail & Related papers  (2023-05-04T19:02:29Z)
• Conversational Text-to-SQL: An Odyssey into State-of-the-Art and Challenges Ahead [6.966624873109535]
  State-of-the-art (SOTA) systems use large, pre-trained and finetuned language models, such as the T5-family. With multi-tasking (MT) over coherent tasks with discrete prompts during training, we improve over specialized text-to-three models. We conduct studies to tease apart errors attributable to domain and compositional generalization.
  arXiv  Detail & Related papers  (2023-02-21T23:15:33Z)
• Know What I don't Know: Handling Ambiguous and Unanswerable Questions for Text-to-SQL [36.5089235153207]
  Existing text-to-yourselfs generate a "plausible" query for an arbitrary user question. We propose a simple yet effective generation approach that automatically produces ambiguous and unanswerable examples. Experimental results show that our model achieves the best result on both real-world examples and generated examples.
  arXiv  Detail & Related papers  (2022-12-17T15:32:00Z)
• A Survey on Text-to-SQL Parsing: Concepts, Methods, and Future Directions [102.8606542189429]
  The goal of text-to-corpora parsing is to convert a natural language (NL) question to its corresponding structured query language () based on the evidences provided by databases. Deep neural networks have significantly advanced this task by neural generation models, which automatically learn a mapping function from an input NL question to an output query.
  arXiv  Detail & Related papers  (2022-08-29T14:24:13Z)
• Weakly Supervised Text-to-SQL Parsing through Question Decomposition [53.22128541030441]
  We take advantage of the recently proposed question meaning representation called QDMR. Given questions, their QDMR structures (annotated by non-experts or automatically predicted) and the answers, we are able to automatically synthesizesql queries. Our results show that the weakly supervised models perform competitively with those trained on NL- benchmark data.
  arXiv  Detail & Related papers  (2021-12-12T20:02:42Z)
• Dual Reader-Parser on Hybrid Textual and Tabular Evidence for Open Domain Question Answering [78.9863753810787]
  A large amount of world's knowledge is stored in structured databases. query languages can answer questions that require complex reasoning, as well as offering full explainability.
  arXiv  Detail & Related papers  (2021-08-05T22:04:13Z)
• "What Do You Mean by That?" A Parser-Independent Interactive Approach for Enhancing Text-to-SQL [49.85635994436742]
  We include human in the loop and present a novel-independent interactive approach (PIIA) that interacts with users using multi-choice questions. PIIA is capable of enhancing the text-to-domain performance with limited interaction turns by using both simulation and human evaluation.
  arXiv  Detail & Related papers  (2020-11-09T02:14:33Z)
• Did You Ask a Good Question? A Cross-Domain Question Intention Classification Benchmark for Text-to-SQL [32.946103197082124]
  Triage is the first cross-domain text-to-question classification benchmark. It requires models to distinguish four types of unanswerable questions from answerable questions. The baseline RoBERTa model achieves a 60% F1 score on the test set.
  arXiv  Detail & Related papers  (2020-10-23T19:36:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.