Grounding Natural Language to SQL Translation with Data-Based Self-Explanations

• URL: http://arxiv.org/abs/2411.02948v1
• Date: Tue, 05 Nov 2024 09:44:53 GMT
• Title: Grounding Natural Language to SQL Translation with Data-Based Self-Explanations
• Authors: Yuankai Fan, Tonghui Ren, Can Huang, Zhenying He, X. Sean Wang,
• Abstract summary: Cycle is a framework designed for end-to-end translation models to autonomously generate the best output through self-evaluation. The main idea is to introduce data-grounded NL explanations as self-provided feedback, and use the feedback to validate the correctness of translation. The results show that 1) the feedback loop introduced in Cycle can consistently improve the performance of existing models, and in particular, by applying Cycle to RESD, obtains a translation accuracy of 82.0% (+2.6%) on the validation set, and 81.6% (+3.2%) on the test set benchmark.
• Score: 7.4643285253289475
• License:
• Abstract: Natural Language Interfaces for Databases empower non-technical users to interact with data using natural language (NL). Advanced approaches, utilizing either neural sequence-to-sequence or more recent sophisticated large-scale language models, typically implement NL to SQL (NL2SQL) translation in an end-to-end fashion. However, like humans, these end-to-end translation models may not always generate the best SQL output on their first try. In this paper, we propose CycleSQL, an iterative framework designed for end-to-end translation models to autonomously generate the best output through self-evaluation. The main idea of CycleSQL is to introduce data-grounded NL explanations of query results as self-provided feedback, and use the feedback to validate the correctness of the translation iteratively, hence improving the overall translation accuracy. Extensive experiments, including quantitative and qualitative evaluations, are conducted to study CycleSQL by applying it to seven existing translation models on five widely used benchmarks. The results show that 1) the feedback loop introduced in CycleSQL can consistently improve the performance of existing models, and in particular, by applying CycleSQL to RESDSQL, obtains a translation accuracy of 82.0% (+2.6%) on the validation set, and 81.6% (+3.2%) on the test set of Spider benchmark; 2) the generated NL explanations can also provide insightful information for users, aiding in the comprehension of translation results and consequently enhancing the interpretability of NL2SQL translation.

Related papers

• Enhancing LLM Fine-tuning for Text-to-SQLs by SQL Quality Measurement [1.392448435105643]
  Text-to-s enables non-expert users to effortlessly retrieve desired information from databases using natural language queries. Current state-of-the-art (SOTA) models like GPT4 and T5 have shown impressive performance on large-scale benchmarks like BIRD. This paper proposed a novel approach that only needs SQL Quality to enhance Text-to-s performance.
  arXiv  Detail & Related papers  (2024-10-02T17:21:51Z)
• Metasql: A Generate-then-Rank Framework for Natural Language to SQL Translation [10.812409371488913]
  We propose a unified generate-then-rank framework that can be flexibly incorporated with existing NLIDBs to consistently improve translation accuracy. Metasql introduces query metadata to control the generation of better query candidates and uses learning-to-rank algorithms to retrieve globally optimized queries. The results show that the performance of the translation models can be effectively improved using Metasql.
  arXiv  Detail & Related papers  (2024-02-27T02:16:07Z)
• Ar-Spider: Text-to-SQL in Arabic [11.463438573648297]
  This paper introduces Ar-Spider 1, the first Arabic cross-language text-to-domain dataset. Due to the unique nature of the language, two major challenges have been encountered, namely linguistic and structural challenges. We propose the similarity relationship (CSR) approach, which results in a significant increase in the overall performance of about 1.52% for S2 and 1.06% for LGE and closes the gap between Arabic and English languages to 7.73%.
  arXiv  Detail & Related papers  (2024-02-22T23:11:17Z)
• Natural language to SQL in low-code platforms [0.0]
  We propose a pipeline allowing developers to write natural language (NL) queries. We collect, label, and validate data covering the queries most often performed by OutSystems users. We describe the entire pipeline, which comprises a feedback loop that allows us to quickly collect production data.
  arXiv  Detail & Related papers  (2023-08-29T11:59:02Z)
• Improving Domain-Specific Retrieval by NLI Fine-Tuning [64.79760042717822]
  This article investigates the fine-tuning potential of natural language inference (NLI) data to improve information retrieval and ranking. We employ both monolingual and multilingual sentence encoders fine-tuned by a supervised method utilizing contrastive loss and NLI data. Our results point to the fact that NLI fine-tuning increases the performance of the models in both tasks and both languages, with the potential to improve mono- and multilingual models.
  arXiv  Detail & Related papers  (2023-08-06T12:40:58Z)
• Interleaving Pre-Trained Language Models and Large Language Models for Zero-Shot NL2SQL Generation [23.519727682763644]
  ZeroNL2 is crucial in achieving natural language tosql that is adaptive to new environments. Existing approaches either fine-tune pretrained language models (PLMs) based on data or use prompts to guide fixed large language models (LLMs) such as ChatGPT. We propose a ZeroNL2 framework that combines the complementary advantages of PLMs and LLMs for supporting zero-shot NL2.
  arXiv  Detail & Related papers  (2023-06-15T06:50:51Z)
• SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL (extended) [53.95151604061761]
  This paper introduces the framework for enhancing Text-to- filtering using large language models (LLMs) With few-shot prompting, we explore the effectiveness of consistency decoding with execution-based error analyses. With instruction fine-tuning, we delve deep in understanding the critical paradigms that influence the performance of tuned LLMs.
  arXiv  Detail & Related papers  (2023-05-26T21:39:05Z)
• UNITE: A Unified Benchmark for Text-to-SQL Evaluation [72.72040379293718]
  We introduce a UNIfied benchmark for Text-to-domain systems. It is composed of publicly available text-to-domain datasets and 29K databases. Compared to the widely used Spider benchmark, we introduce a threefold increase in SQL patterns.
  arXiv  Detail & Related papers  (2023-05-25T17:19:52Z)
• Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustness [115.66421993459663]
  Recent studies reveal that text-to- models are vulnerable to task-specific perturbations. We propose a comprehensive robustness benchmark based on Spider to diagnose the model. We conduct a diagnostic study of the state-of-the-art models on the set.
  arXiv  Detail & Related papers  (2023-01-21T03:57:18Z)
• mFACE: Multilingual Summarization with Factual Consistency Evaluation [79.60172087719356]
  Abstractive summarization has enjoyed renewed interest in recent years, thanks to pre-trained language models and the availability of large-scale datasets. Despite promising results, current models still suffer from generating factually inconsistent summaries. We leverage factual consistency evaluation models to improve multilingual summarization.
  arXiv  Detail & Related papers  (2022-12-20T19:52:41Z)
• XRICL: Cross-lingual Retrieval-Augmented In-Context Learning for Cross-lingual Text-to-SQL Semantic Parsing [70.40401197026925]
  In-context learning using large language models has recently shown surprising results for semantic parsing tasks. This work introduces the XRICL framework, which learns to retrieve relevant English exemplars for a given query. We also include global translation exemplars for a target language to facilitate the translation process for large language models.
  arXiv  Detail & Related papers  (2022-10-25T01:33:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.