A Neural Question Answering System for Basic Questions about Subroutines

• URL: http://arxiv.org/abs/2101.03999v1
• Date: Mon, 11 Jan 2021 16:18:52 GMT
• Title: A Neural Question Answering System for Basic Questions about Subroutines
• Authors: Aakash Bansal, Zachary Eberhart, Lingfei Wu, Collin McMillan
• Abstract summary: A question answering (QA) system is a type of conversational AI that generates natural language answers to questions posed by human users. We take initial steps to bringing state-of-the-art neural technologies to Software Engineering applications by designing a context-based QA system.
• Score: 32.98355443690782
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A question answering (QA) system is a type of conversational AI that generates natural language answers to questions posed by human users. QA systems often form the backbone of interactive dialogue systems, and have been studied extensively for a wide variety of tasks ranging from restaurant recommendations to medical diagnostics. Dramatic progress has been made in recent years, especially from the use of encoder-decoder neural architectures trained with big data input. In this paper, we take initial steps to bringing state-of-the-art neural QA technologies to Software Engineering applications by designing a context-based QA system for basic questions about subroutines. We curate a training dataset of 10.9 million question/context/answer tuples based on rules we extract from recent empirical studies. Then, we train a custom neural QA model with this dataset and evaluate the model in a study with professional programmers. We demonstrate the strengths and weaknesses of the system, and lay the groundwork for its use in eventual dialogue systems for software engineering.

Related papers

• A Joint-Reasoning based Disease Q&A System [6.117758142183177]
  Medical question answer (QA) assistants respond to lay users' health-related queries by synthesizing information from multiple sources. They can serve as vital tools to alleviate issues of misinformation, information overload, and complexity of medical language.
  arXiv  Detail & Related papers  (2024-01-06T09:55:22Z)
• PIE-QG: Paraphrased Information Extraction for Unsupervised Question Generation from Small Corpora [4.721845865189576]
  PIE-QG uses Open Information Extraction (OpenIE) to generate synthetic training questions from paraphrased passages. Triples in the form of subject, predicate, object> are extracted from each passage, and questions are formed with subjects (or objects) and predicates while objects (or subjects) are considered as answers.
  arXiv  Detail & Related papers  (2023-01-03T12:20:51Z)
• Evaluation of Question Answering Systems: Complexity of judging a natural language [3.4771957347698583]
  Question answering (QA) systems are among the most important and rapidly developing research topics in natural language processing (NLP) This survey attempts to provide a systematic overview of the general framework of QA, QA paradigms, benchmark datasets, and assessment techniques for a quantitative evaluation of QA systems.
  arXiv  Detail & Related papers  (2022-09-10T12:29:04Z)
• Improving Unsupervised Question Answering via Summarization-Informed Question Generation [47.96911338198302]
  Question Generation (QG) is the task of generating a plausible question for a passage, answer> pair. We make use of freely available news summary data, transforming declarative sentences into appropriate questions using dependency parsing, named entity recognition and semantic role labeling. The resulting questions are then combined with the original news articles to train an end-to-end neural QG model.
  arXiv  Detail & Related papers  (2021-09-16T13:08:43Z)
• Solving Machine Learning Problems [0.315565869552558]
  This work trains a machine learning model to solve machine learning problems from a University undergraduate level course. We generate a new training set of questions and answers consisting of course exercises, homework, and quiz questions from MIT's 6.036 Introduction to Machine Learning course. Our system demonstrates an overall accuracy of 96% for open-response questions and 97% for multiple-choice questions, compared with MIT students' average of 93%.
  arXiv  Detail & Related papers  (2021-07-02T18:52:50Z)
• BERT-CoQAC: BERT-based Conversational Question Answering in Context [10.811729691130349]
  We introduce a framework based on a publically available pre-trained language model called BERT for incorporating history turns into the system. Experiment results revealed that our framework is comparable in performance with the state-of-the-art models on the QuAC leader board.
  arXiv  Detail & Related papers  (2021-04-23T03:05:17Z)
• Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answering [62.88322725956294]
  We review the latest research trends in OpenQA, with particular attention to systems that incorporate neural MRC techniques. We introduce modern OpenQA architecture named Retriever-Reader'' and analyze the various systems that follow this architecture. We then discuss key challenges to developing OpenQA systems and offer an analysis of benchmarks that are commonly used.
  arXiv  Detail & Related papers  (2021-01-04T04:47:46Z)
• Question Answering over Knowledge Bases by Leveraging Semantic Parsing and Neuro-Symbolic Reasoning [73.00049753292316]
  We propose a semantic parsing and reasoning-based Neuro-Symbolic Question Answering(NSQA) system. NSQA achieves state-of-the-art performance on QALD-9 and LC-QuAD 1.0.
  arXiv  Detail & Related papers  (2020-12-03T05:17:55Z)
• Few-Shot Complex Knowledge Base Question Answering via Meta Reinforcement Learning [55.08037694027792]
  Complex question-answering (CQA) involves answering complex natural-language questions on a knowledge base (KB) The conventional neural program induction (NPI) approach exhibits uneven performance when the questions have different types. This paper proposes a meta-reinforcement learning approach to program induction in CQA to tackle the potential distributional bias in questions.
  arXiv  Detail & Related papers  (2020-10-29T18:34:55Z)
• Retrieve, Program, Repeat: Complex Knowledge Base Question Answering via Alternate Meta-learning [56.771557756836906]
  We present a novel method that automatically learns a retrieval model alternately with the programmer from weak supervision. Our system leads to state-of-the-art performance on a large-scale task for complex question answering over knowledge bases.
  arXiv  Detail & Related papers  (2020-10-29T18:28:16Z)
• Interpretable Multi-Step Reasoning with Knowledge Extraction on Complex Healthcare Question Answering [89.76059961309453]
  HeadQA dataset contains multiple-choice questions authorized for the public healthcare specialization exam. These questions are the most challenging for current QA systems. We present a Multi-step reasoning with Knowledge extraction framework (MurKe) We are striving to make full use of off-the-shelf pre-trained models.
  arXiv  Detail & Related papers  (2020-08-06T02:47:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.