New Methods & Metrics for LFQA tasks

• URL: http://arxiv.org/abs/2112.13432v1
• Date: Sun, 26 Dec 2021 18:38:05 GMT
• Title: New Methods & Metrics for LFQA tasks
• Authors: Suchismit Mahapatra, Vladimir Blagojevic, Pablo Bertorello, Prasanna Kumar
• Abstract summary: Long-form question answering tasks require retrieving the documents pertinent to a query, using them to form a paragraph-length answer. This work addresses the train/validation/test dataset overlap, absence of automatic metrics and generated answers not being "grounded" in retrieved documents.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Long-form question answering (LFQA) tasks require retrieving the documents pertinent to a query, using them to form a paragraph-length answer. Despite considerable progress in LFQA modeling, fundamental issues impede its progress: i) train/validation/test dataset overlap, ii) absence of automatic metrics and iii) generated answers not being "grounded" in retrieved documents. This work addresses every one these critical bottlenecks, contributing natural language inference/generation (NLI/NLG) methods and metrics that make significant strides to their alleviation.

Related papers

• Exploring Hint Generation Approaches in Open-Domain Question Answering [16.434748534272014]
  We introduce a novel context preparation approach called HINTQA. Unlike traditional methods, HINTQA prompts LLMs to produce hints about potential answers for the question. We demonstrate that hints enhance the accuracy of answers more than retrieved and generated contexts.
  arXiv  Detail & Related papers  (2024-09-24T13:50:32Z)
• MFORT-QA: Multi-hop Few-shot Open Rich Table Question Answering [3.1651118728570635]
  In today's fast-paced industry, professionals face the challenge of summarizing a large number of documents and extracting vital information from them on a daily basis. To address this challenge, the approach of Table Question Answering (QA) has been developed to extract the relevant information. Recent advancements in Large Language Models (LLMs) have opened up new possibilities for extracting information from tabular data using prompts.
  arXiv  Detail & Related papers  (2024-03-28T03:14:18Z)
• Automatic Question-Answer Generation for Long-Tail Knowledge [65.11554185687258]
  We propose an automatic approach to generate specialized QA datasets for tail entities. We conduct extensive experiments by employing pretrained LLMs on our newly generated long-tail QA datasets.
  arXiv  Detail & Related papers  (2024-03-03T03:06:31Z)
• Long-form Question Answering: An Iterative Planning-Retrieval-Generation Approach [28.849548176802262]
  Long-form question answering (LFQA) poses a challenge as it involves generating detailed answers in the form of paragraphs. We propose an LFQA model with iterative Planning, Retrieval, and Generation. We find that our model outperforms the state-of-the-art models on various textual and factual metrics for the LFQA task.
  arXiv  Detail & Related papers  (2023-11-15T21:22:27Z)
• Event Extraction as Question Generation and Answering [72.04433206754489]
  Recent work on Event Extraction has reframed the task as Question Answering (QA) We propose QGA-EE, which enables a Question Generation (QG) model to generate questions that incorporate rich contextual information instead of using fixed templates. Experiments show that QGA-EE outperforms all prior single-task-based models on the ACE05 English dataset.
  arXiv  Detail & Related papers  (2023-07-10T01:46:15Z)
• Self-Prompting Large Language Models for Zero-Shot Open-Domain QA [67.08732962244301]
  Open-Domain Question Answering (ODQA) aims to answer questions without explicitly providing background documents. This task becomes notably challenging in a zero-shot setting where no data is available to train tailored retrieval-reader models. We propose a Self-Prompting framework to explicitly utilize the massive knowledge encoded in the parameters of Large Language Models.
  arXiv  Detail & Related papers  (2022-12-16T18:23:43Z)
• Questions Are All You Need to Train a Dense Passage Retriever [123.13872383489172]
  ART is a new corpus-level autoencoding approach for training dense retrieval models that does not require any labeled training data. It uses a new document-retrieval autoencoding scheme, where (1) an input question is used to retrieve a set of evidence documents, and (2) the documents are then used to compute the probability of reconstructing the original question.
  arXiv  Detail & Related papers  (2022-06-21T18:16:31Z)
• ASQA: Factoid Questions Meet Long-Form Answers [35.11889930792675]
  This work focuses on factoid questions that are ambiguous, that is, have different correct answers depending on interpretation. Answers to ambiguous questions should synthesize factual information from multiple sources into a long-form summary. We use this notion of correctness to define an automated metric of performance for ASQA.
  arXiv  Detail & Related papers  (2022-04-12T21:58:44Z)
• Read before Generate! Faithful Long Form Question Answering with Machine Reading [77.17898499652306]
  Long-form question answering (LFQA) aims to generate a paragraph-length answer for a given question. We propose a new end-to-end framework that jointly models answer generation and machine reading.
  arXiv  Detail & Related papers  (2022-03-01T10:41:17Z)
• Hurdles to Progress in Long-form Question Answering [34.805039943215284]
  We show that the task formulation raises fundamental challenges regarding evaluation and dataset creation. We first design a new system that relies on sparse attention and contrastive retriever learning to achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2021-03-10T20:32:30Z)
• Open Question Answering over Tables and Text [55.8412170633547]
  In open question answering (QA), the answer to a question is produced by retrieving and then analyzing documents that might contain answers to the question. Most open QA systems have considered only retrieving information from unstructured text. We present a new large-scale dataset Open Table-and-Text Question Answering (OTT-QA) to evaluate performance on this task.
  arXiv  Detail & Related papers  (2020-10-20T16:48:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.