Evidentiality-aware Retrieval for Overcoming Abstractiveness in Open-Domain Question Answering

• URL: http://arxiv.org/abs/2304.03031v6
• Date: Thu, 1 Feb 2024 17:36:39 GMT
• Title: Evidentiality-aware Retrieval for Overcoming Abstractiveness in Open-Domain Question Answering
• Authors: Yongho Song, Dahyun Lee, Myungha Jang, Seung-won Hwang, Kyungjae Lee, Dongha Lee, Jinyeong Yeo
• Abstract summary: We propose Evidentiality-Aware Passage Retrieval (EADPR) to learn to discriminate evidence passages from distractors. We conduct extensive experiments to validate the effectiveness of our proposed method on multiple abstractive ODQA tasks.
• Score: 29.00167886463793
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The long-standing goal of dense retrievers in abtractive open-domain question answering (ODQA) tasks is to learn to capture evidence passages among relevant passages for any given query, such that the reader produce factually correct outputs from evidence passages. One of the key challenge is the insufficient amount of training data with the supervision of the answerability of the passages. Recent studies rely on iterative pipelines to annotate answerability using signals from the reader, but their high computational costs hamper practical applications. In this paper, we instead focus on a data-centric approach and propose Evidentiality-Aware Dense Passage Retrieval (EADPR), which leverages synthetic distractor samples to learn to discriminate evidence passages from distractors. We conduct extensive experiments to validate the effectiveness of our proposed method on multiple abstractive ODQA tasks.

Related papers

• Passage-specific Prompt Tuning for Passage Reranking in Question Answering with Large Language Models [11.716595438057997]
  We propose passage-specific prompt tuning for reranking in open-domain question answering (PSPT) PSPT is a parameter-efficient method that fine-tunes learnable passage-specific soft prompts. We conducted extensive experiments utilizing the Llama-2-chat-7B model across three publicly available open-domain question answering datasets.
  arXiv  Detail & Related papers  (2024-05-31T07:43:42Z)
• Improving Zero-shot Reader by Reducing Distractions from Irrelevant Documents in Open-Domain Question Answering [10.794156033638984]
  Large language models (LLMs) enable zero-shot approaches in open-domain question answering (ODQA) This study aims at the feasibility of a zero-shot reader that addresses the challenges of computational cost and the need for labeled data.
  arXiv  Detail & Related papers  (2023-10-26T15:45:12Z)
• Strong and Efficient Baselines for Open Domain Conversational Question Answering [2.773656427800412]
  We study the State-of-the-Art (SotA) Dense Passage Retrieval (DPR) retriever and Fusion-in-Decoder (FiD) reader pipeline. We propose and evaluate strong yet simple and efficient baselines, by introducing a fast reranking component between the retriever and the reader. Experiments on two ODConvQA tasks, namely TopiOCQA and OR-QuAC, show that our method improves the SotA results, while reducing reader's latency by 60%.
  arXiv  Detail & Related papers  (2023-10-23T08:48:14Z)
• On the Universal Adversarial Perturbations for Efficient Data-free Adversarial Detection [55.73320979733527]
  We propose a data-agnostic adversarial detection framework, which induces different responses between normal and adversarial samples to UAPs. Experimental results show that our method achieves competitive detection performance on various text classification tasks.
  arXiv  Detail & Related papers  (2023-06-27T02:54:07Z)
• Passage-Mask: A Learnable Regularization Strategy for Retriever-Reader Models [36.58955176223759]
  Retriever-reader models achieve competitive performance across many different NLP tasks such as open question answering and dialogue conversations. We introduce a learnable passage mask mechanism which desensitizes the impact from the top-rank retrieval passages and prevents the model from overfitting.
  arXiv  Detail & Related papers  (2022-11-02T06:39:46Z)
• PReGAN: Answer Oriented Passage Ranking with Weakly Supervised GAN [34.96355889356033]
  We propose an approach called tttPReGAN for Passage Reranking based on Generative Adversarial Neural networks. The goal is to force the generator to rank higher a passage that is topically relevant and contains an answer.
  arXiv  Detail & Related papers  (2022-07-05T01:43:35Z)
• A Unified End-to-End Retriever-Reader Framework for Knowledge-based VQA [67.75989848202343]
  This paper presents a unified end-to-end retriever-reader framework towards knowledge-based VQA. We shed light on the multi-modal implicit knowledge from vision-language pre-training models to mine its potential in knowledge reasoning. Our scheme is able to not only provide guidance for knowledge retrieval, but also drop these instances potentially error-prone towards question answering.
  arXiv  Detail & Related papers  (2022-06-30T02:35:04Z)
• Joint Passage Ranking for Diverse Multi-Answer Retrieval [56.43443577137929]
  We study multi-answer retrieval, an under-explored problem that requires retrieving passages to cover multiple distinct answers for a question. This task requires joint modeling of retrieved passages, as models should not repeatedly retrieve passages containing the same answer at the cost of missing a different valid answer. In this paper, we introduce JPR, a joint passage retrieval model focusing on reranking. To model the joint probability of the retrieved passages, JPR makes use of an autoregressive reranker that selects a sequence of passages, equipped with novel training and decoding algorithms.
  arXiv  Detail & Related papers  (2021-04-17T04:48:36Z)
• A Graph-guided Multi-round Retrieval Method for Conversational Open-domain Question Answering [52.041815783025186]
  We propose a novel graph-guided retrieval method to model the relations among answers across conversation turns. We also propose to incorporate the multi-round relevance feedback technique to explore the impact of the retrieval context on current question understanding.
  arXiv  Detail & Related papers  (2021-04-17T04:39:41Z)
• Complementary Evidence Identification in Open-Domain Question Answering [66.17954897343456]
  We propose a new problem of complementary evidence identification for open-domain question answering (QA) The problem aims to efficiently find a small set of passages that covers full evidence from multiple aspects as to answer a complex question.
  arXiv  Detail & Related papers  (2021-03-22T08:04:50Z)
• Mining Implicit Relevance Feedback from User Behavior for Web Question Answering [92.45607094299181]
  We make the first study to explore the correlation between user behavior and passage relevance. Our approach significantly improves the accuracy of passage ranking without extra human labeled data. In practice, this work has proved effective to substantially reduce the human labeling cost for the QA service in a global commercial search engine.
  arXiv  Detail & Related papers  (2020-06-13T07:02:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.