Few-shot In-context Learning for Knowledge Base Question Answering

• URL: http://arxiv.org/abs/2305.01750v2
• Date: Thu, 4 May 2023 14:50:38 GMT
• Title: Few-shot In-context Learning for Knowledge Base Question Answering
• Authors: Tianle Li, Xueguang Ma, Alex Zhuang, Yu Gu, Yu Su and Wenhu Chen
• Abstract summary: We propose KB-BINDER, which for the first time enables few-shot in-context learning over KBQA tasks. The experimental results on four public heterogeneous KBQA datasets show that KB-BINDER can achieve a strong performance with only a few in-context demonstrations.
• Score: 31.73274700847965
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Question answering over knowledge bases is considered a difficult problem due to the challenge of generalizing to a wide variety of possible natural language questions. Additionally, the heterogeneity of knowledge base schema items between different knowledge bases often necessitates specialized training for different knowledge base question-answering (KBQA) datasets. To handle questions over diverse KBQA datasets with a unified training-free framework, we propose KB-BINDER, which for the first time enables few-shot in-context learning over KBQA tasks. Firstly, KB-BINDER leverages large language models like Codex to generate logical forms as the draft for a specific question by imitating a few demonstrations. Secondly, KB-BINDER grounds on the knowledge base to bind the generated draft to an executable one with BM25 score matching. The experimental results on four public heterogeneous KBQA datasets show that KB-BINDER can achieve a strong performance with only a few in-context demonstrations. Especially on GraphQA and 3-hop MetaQA, KB-BINDER can even outperform the state-of-the-art trained models. On GrailQA and WebQSP, our model is also on par with other fully-trained models. We believe KB-BINDER can serve as an important baseline for future research. Our code is available at https://github.com/ltl3A87/KB-BINDER.

Related papers

• A Learn-Then-Reason Model Towards Generalization in Knowledge Base Question Answering [17.281005999581865]
  Large-scale knowledge bases (KBs) like Freebase and Wikidata house millions of structured knowledge. Knowledge Base Question Answering (KBQA) provides a user-friendly way to access these valuable KBs via asking natural language questions. This paper develops KBLLaMA, which follows a learn-then-reason framework to inject new KB knowledge into a large language model for flexible end-to-end KBQA.
  arXiv  Detail & Related papers  (2024-06-20T22:22:41Z)
• Few-shot Transfer Learning for Knowledge Base Question Answering: Fusing Supervised Models with In-Context Learning [20.80841972133938]
  Existing Knowledge Base Question Answering (KBQA) architectures are hungry for annotated data. We introduce the problem of few-shot transfer learning for KBQA, where the target domain offers only a few labeled examples. We propose a novel KBQA architecture called FuSIC-KBQA that performs KB-retrieval using multiple source-trained retrievers.
  arXiv  Detail & Related papers  (2023-11-15T11:56:56Z)
• ChatKBQA: A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Models [19.85526116658481]
  We introduce ChatKBQA, a novel and simple generate-then-retrieve KBQA framework. Experimental results show that ChatKBQA achieves new state-of-the-art performance on standard KBQA datasets. This work can also be regarded as a new paradigm for combining LLMs with knowledge graphs for interpretable and knowledge-required question answering.
  arXiv  Detail & Related papers  (2023-10-13T09:45:14Z)
• Open-Set Knowledge-Based Visual Question Answering with Inference Paths [79.55742631375063]
  The purpose of Knowledge-Based Visual Question Answering (KB-VQA) is to provide a correct answer to the question with the aid of external knowledge bases. We propose a new retriever-ranker paradigm of KB-VQA, Graph pATH rankER (GATHER for brevity) Specifically, it contains graph constructing, pruning, and path-level ranking, which not only retrieves accurate answers but also provides inference paths that explain the reasoning process.
  arXiv  Detail & Related papers  (2023-10-12T09:12:50Z)
• Make a Choice! Knowledge Base Question Answering with In-Context Learning [1.7827767384590838]
  Question answering over knowledge bases (KBQA) aims to answer factoid questions with a given knowledge base (KB) Due to the large scale of KB, annotated data is impossible to cover all fact schemas in KB. We present McL-KBQA, a framework that incorporates the few-shot ability of LLM into the KBQA method via ICL-based multiple choice.
  arXiv  Detail & Related papers  (2023-05-23T11:56:03Z)
• Cross-Lingual Question Answering over Knowledge Base as Reading Comprehension [61.079852289005025]
  Cross-lingual question answering over knowledge base (xKBQA) aims to answer questions in languages different from that of the provided knowledge base. One of the major challenges facing xKBQA is the high cost of data annotation. We propose a novel approach for xKBQA in a reading comprehension paradigm.
  arXiv  Detail & Related papers  (2023-02-26T05:52:52Z)
• QA Is the New KR: Question-Answer Pairs as Knowledge Bases [105.692569000534]
  We argue that the proposed type of KB has many of the key advantages of a traditional symbolic KB. Unlike a traditional KB, this information store is well-aligned with common user information needs.
  arXiv  Detail & Related papers  (2022-07-01T19:09:08Z)
• SYGMA: System for Generalizable Modular Question Answering OverKnowledge Bases [57.89642289610301]
  We present SYGMA, a modular approach facilitating general-izability across multiple knowledge bases and multiple rea-soning types. We demonstrate effectiveness of our system by evaluating on datasets belonging to two distinct knowledge bases,DBpedia and Wikidata.
  arXiv  Detail & Related papers  (2021-09-28T01:57:56Z)
• Reasoning Over Virtual Knowledge Bases With Open Predicate Relations [85.19305347984515]
  We present the Open Predicate Query Language (OPQL) OPQL is a method for constructing a virtual Knowledge Base (VKB) trained entirely from text. We demonstrate that OPQL outperforms prior VKB methods on two different KB reasoning tasks.
  arXiv  Detail & Related papers  (2021-02-14T01:29:54Z)
• Beyond I.I.D.: Three Levels of Generalization for Question Answering on Knowledge Bases [63.43418760818188]
  We release a new large-scale, high-quality dataset with 64,331 questions, GrailQA. We propose a novel BERT-based KBQA model. The combination of our dataset and model enables us to thoroughly examine and demonstrate, for the first time, the key role of pre-trained contextual embeddings like BERT in the generalization of KBQA.
  arXiv  Detail & Related papers  (2020-11-16T06:36:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.