KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering

• URL: http://arxiv.org/abs/2110.04330v1
• Date: Fri, 8 Oct 2021 18:39:59 GMT
• Title: KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering
• Authors: Donghan Yu, Chenguang Zhu, Yuwei Fang, Wenhao Yu, Shuohang Wang, Yichong Xu, Xiang Ren, Yiming Yang, Michael Zeng
• Abstract summary: We propose a novel method KG-FiD, which filters noisy passages by leveraging the structural relationship among the retrieved passages with a knowledge graph. We show that KG-FiD can improve vanilla FiD by up to 1.5% on answer exact match score and achieve comparable performance with FiD with only 40% of computation cost.
• Score: 68.00631278030627
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Current Open-Domain Question Answering (ODQA) model paradigm often contains a retrieving module and a reading module. Given an input question, the reading module predicts the answer from the relevant passages which are retrieved by the retriever. The recent proposed Fusion-in-Decoder (FiD), which is built on top of the pretrained generative model T5, achieves the state-of-the-art performance in the reading module. Although being effective, it remains constrained by inefficient attention on all retrieved passages which contain a lot of noise. In this work, we propose a novel method KG-FiD, which filters noisy passages by leveraging the structural relationship among the retrieved passages with a knowledge graph. We initiate the passage node embedding from the FiD encoder and then use graph neural network (GNN) to update the representation for reranking. To improve the efficiency, we build the GNN on top of the intermediate layer output of the FiD encoder and only pass a few top reranked passages into the higher layers of encoder and decoder for answer generation. We also apply the proposed GNN based reranking method to enhance the passage retrieval results in the retrieving module. Extensive experiments on common ODQA benchmark datasets (Natural Question and TriviaQA) demonstrate that KG-FiD can improve vanilla FiD by up to 1.5% on answer exact match score and achieve comparable performance with FiD with only 40% of computation cost.

Related papers

• FastFiD: Improve Inference Efficiency of Open Domain Question Answering via Sentence Selection [61.9638234358049]
  FastFiD is a novel approach that executes sentence selection on encoded passages. This aids in retaining valuable sentences while reducing the context length required for generating answers.
  arXiv  Detail & Related papers  (2024-08-12T17:50:02Z)
• Graph Neural Network Enhanced Retrieval for Question Answering of LLMs [19.24603296717601]
  Existing retrieval methods divide reference documents into passages, treating them in isolation. These passages, however, are often interrelated, such as passages that are contiguous or share the same keywords. We propose a novel retrieval method, called GNN-Ret, which leverages graph neural networks (GNNs) to enhance retrieval by exploiting the relatedness between passages.
  arXiv  Detail & Related papers  (2024-06-03T17:07:46Z)
• ReasoningLM: Enabling Structural Subgraph Reasoning in Pre-trained Language Models for Question Answering over Knowledge Graph [142.42275983201978]
  We propose a subgraph-aware self-attention mechanism to imitate the GNN for performing structured reasoning. We also adopt an adaptation tuning strategy to adapt the model parameters with 20,000 subgraphs with synthesized questions. Experiments show that ReasoningLM surpasses state-of-the-art models by a large margin, even with fewer updated parameters and less training data.
  arXiv  Detail & Related papers  (2023-12-30T07:18:54Z)
• Decoupled DETR For Few-shot Object Detection [4.520231308678286]
  We improve the FSOD model to address the severe issue of sample imbalance and weak feature propagation. We build a unified decoder module that could dynamically fuse the decoder layers as the output feature. Our results indicate that our proposed module could achieve stable improvements of 5% to 10% in both fine-tuning and meta-learning paradigms.
  arXiv  Detail & Related papers  (2023-11-20T07:10:39Z)
• Noisy Pair Corrector for Dense Retrieval [59.312376423104055]
  We propose a novel approach called Noisy Pair Corrector (NPC) NPC consists of a detection module and a correction module. We conduct experiments on text-retrieval benchmarks Natural Question and TriviaQA, code-search benchmarks StaQC and SO-DS.
  arXiv  Detail & Related papers  (2023-11-07T08:27:14Z)
• RFiD: Towards Rational Fusion-in-Decoder for Open-Domain Question Answering [11.62870729875824]
  Open-Domain Question Answering (ODQA) systems necessitate a reader model capable of generating answers by simultaneously referring to multiple passages. We introduce the Rational Fusion-in-Decoder (RFiD) model to counter this problem.
  arXiv  Detail & Related papers  (2023-05-26T15:51:25Z)
• Cost Aggregation Is All You Need for Few-Shot Segmentation [28.23753949369226]
  We introduce Volumetric Aggregation with Transformers (VAT) to tackle the few-shot segmentation task. VAT uses both convolutions and transformers to efficiently handle high dimensional correlation maps between query and support. We find that the proposed method attains state-of-the-art performance even for the standard benchmarks in semantic correspondence task.
  arXiv  Detail & Related papers  (2021-12-22T06:18:51Z)
• iffDetector: Inference-aware Feature Filtering for Object Detection [70.8678270164057]
  We introduce a generic Inference-aware Feature Filtering (IFF) module that can easily be combined with modern detectors. IFF performs closed-loop optimization by leveraging high-level semantics to enhance the convolutional features. IFF can be fused with CNN-based object detectors in a plug-and-play manner with negligible computational cost overhead.
  arXiv  Detail & Related papers  (2020-06-23T02:57:29Z)
• Toward Subgraph-Guided Knowledge Graph Question Generation with Graph Neural Networks [53.58077686470096]
  Knowledge graph (KG) question generation (QG) aims to generate natural language questions from KGs and target answers. In this work, we focus on a more realistic setting where we aim to generate questions from a KG subgraph and target answers.
  arXiv  Detail & Related papers  (2020-04-13T15:43:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.