AutoQA: From Databases To QA Semantic Parsers With Only Synthetic Training Data

• URL: http://arxiv.org/abs/2010.04806v2
• Date: Tue, 8 Jun 2021 00:49:21 GMT
• Title: AutoQA: From Databases To QA Semantic Parsers With Only Synthetic Training Data
• Abstract summary: AutoQA generates semantics that answer questions on databases with no manual effort. It uses automatic paraphrasing combined with template-based attribute parsing to find alternative expressions of an expression. It achieves 69.8% answer accuracy, 16.4% higher than the state-of-the-art zero-shot models and only 5.2% lower than the same model trained with human data.
• Score: 6.522647000820583
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose AutoQA, a methodology and toolkit to generate semantic parsers that answer questions on databases, with no manual effort. Given a database schema and its data, AutoQA automatically generates a large set of high-quality questions for training that covers different database operations. It uses automatic paraphrasing combined with template-based parsing to find alternative expressions of an attribute in different parts of speech. It also uses a novel filtered auto-paraphraser to generate correct paraphrases of entire sentences. We apply AutoQA to the Schema2QA dataset and obtain an average logical form accuracy of 62.9% when tested on natural questions, which is only 6.4% lower than a model trained with expert natural language annotations and paraphrase data collected from crowdworkers. To demonstrate the generality of AutoQA, we also apply it to the Overnight dataset. AutoQA achieves 69.8% answer accuracy, 16.4% higher than the state-of-the-art zero-shot models and only 5.2% lower than the same model trained with human data.

Related papers

• AutoDrive-QA- Automated Generation of Multiple-Choice Questions for Autonomous Driving Datasets Using Large Vision-Language Models [1.3812010983144802]
  We introduce AutoDrive-QA, an automatic pipeline that converts existing driving QA datasets into a structured multiple-choice question (MCQ) format. This benchmark systematically assesses perception, prediction, and planning tasks, providing a standardized and objective evaluation framework. We test the benchmark on three public datasets and conduct zero-shot experiments on an unseen dataset.
  arXiv  Detail & Related papers  (2025-03-20T01:32:00Z)
• Long-Span Question-Answering: Automatic Question Generation and QA-System Ranking via Side-by-Side Evaluation [65.16137964758612]
  We explore the use of long-context capabilities in large language models to create synthetic reading comprehension data from entire books. Our objective is to test the capabilities of LLMs to analyze, understand, and reason over problems that require a detailed comprehension of long spans of text.
  arXiv  Detail & Related papers  (2024-05-31T20:15:10Z)
• KET-QA: A Dataset for Knowledge Enhanced Table Question Answering [63.56707527868466]
  We propose to use a knowledge base (KB) as the external knowledge source for TableQA. Every question requires the integration of information from both the table and the sub-graph to be answered. We design a retriever-reasoner structured pipeline model to extract pertinent information from the vast knowledge sub-graph.
  arXiv  Detail & Related papers  (2024-05-13T18:26:32Z)
• A Lightweight Method to Generate Unanswerable Questions in English [18.323248259867356]
  We examine a simpler data augmentation method for unanswerable question generation in English. We perform antonym and entity swaps on answerable questions. Compared to the prior state-of-the-art, data generated with our training-free and lightweight strategy results in better models.
  arXiv  Detail & Related papers  (2023-10-30T10:14:52Z)
• A Practical Toolkit for Multilingual Question and Answer Generation [79.31199020420827]
  We introduce AutoQG, an online service for multilingual QAG, along with lmqg, an all-in-one Python package for model fine-tuning, generation, and evaluation. We also release QAG models in eight languages fine-tuned on a few variants of pre-trained encoder-decoder language models, which can be used online via AutoQG or locally via lmqg.
  arXiv  Detail & Related papers  (2023-05-27T08:42:37Z)
• Parameter-Efficient Abstractive Question Answering over Tables or Text [60.86457030988444]
  A long-term ambition of information seeking QA systems is to reason over multi-modal contexts and generate natural answers to user queries. Memory intensive pre-trained language models are adapted to downstream tasks such as QA by fine-tuning the model on QA data in a specific modality like unstructured text or structured tables. To avoid training such memory-hungry models while utilizing a uniform architecture for each modality, parameter-efficient adapters add and train small task-specific bottle-neck layers between transformer layers.
  arXiv  Detail & Related papers  (2022-04-07T10:56:29Z)
• Unsupervised Evaluation for Question Answering with Transformers [46.16837670041594]
  We investigate the hidden representations of questions, answers, and contexts in transformer-based QA architectures. We observe a consistent pattern in the answer representations, which we show can be used to automatically evaluate whether or not a predicted answer is correct. We are able to predict whether or not a model's answer is correct with 91.37% accuracy SQuAD, and 80.7% accuracy on SubjQA.
  arXiv  Detail & Related papers  (2020-10-07T07:03:30Z)
• Generating Diverse and Consistent QA pairs from Contexts with Information-Maximizing Hierarchical Conditional VAEs [62.71505254770827]
  We propose a conditional variational autoencoder (HCVAE) for generating QA pairs given unstructured texts as contexts. Our model obtains impressive performance gains over all baselines on both tasks, using only a fraction of data for training.
  arXiv  Detail & Related papers  (2020-05-28T08:26:06Z)
• Syntactic Question Abstraction and Retrieval for Data-Scarce Semantic Parsing [22.490892257736043]
  We propose Syntactic Question Abstraction and Retrieval (SQAR) to build a neural semantic that translates a natural language to a logical form. SQAR generates unseen logical patterns upon the addition of new examples without re-training the model. In contrast to a simple pattern classification approach, SQAR can generate unseen logical patterns without re-training the model.
  arXiv  Detail & Related papers  (2020-05-01T23:05:55Z)
• Template-Based Question Generation from Retrieved Sentences for Improved Unsupervised Question Answering [98.48363619128108]
  We propose an unsupervised approach to training QA models with generated pseudo-training data. We show that generating questions for QA training by applying a simple template on a related, retrieved sentence rather than the original context sentence improves downstream QA performance.
  arXiv  Detail & Related papers  (2020-04-24T17:57:45Z)
• Schema2QA: High-Quality and Low-Cost Q&A Agents for the Structured Web [11.387139390711464]
  Building a question-answering agent currently requires large annotated datasets, which are prohibitively expensive. This paper proposes augmented2QA, an open-source toolkit that can generate a Q&A system from a database schema with a few annotations for each field. We use augmented2QA to generate Q&A systems for five.org domains, restaurants, people, movies, books and music, and obtain an overall accuracy between 64% and 75% on crowdsourced questions.
  arXiv  Detail & Related papers  (2020-01-16T01:49:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.