Robust Few-shot Transfer Learning for Knowledge Base Question Answering with Unanswerable Questions

• URL: http://arxiv.org/abs/2406.14313v1
• Date: Thu, 20 Jun 2024 13:43:38 GMT
• Title: Robust Few-shot Transfer Learning for Knowledge Base Question Answering with Unanswerable Questions
• Authors: Riya Sawhney, Indrajit Bhattacharya, Mausam,
• Abstract summary: We present FUn-FuSIC that extends the state-of-the-art (SoTA) few-shot transfer model for answerable-only KBQA to handle unanswerability. Experiments over newly constructed datasets show that FUn-FuSIC outperforms suitable adaptations of the SoTA model for KBQA with unanswerability.
• Score: 22.411601767105807
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Real-world KBQA applications require models that are (1) robust -- e.g., can differentiate between answerable and unanswerable questions, and (2) low-resource -- do not require large training data. Towards this goal, we propose the novel task of few-shot transfer for KBQA with unanswerable questions. We present FUn-FuSIC that extends the state-of-the-art (SoTA) few-shot transfer model for answerable-only KBQA to handle unanswerability. It iteratively prompts an LLM to generate logical forms for the question by providing feedback using a diverse suite of syntactic, semantic and execution guided checks, and adapts self-consistency to assess confidence of the LLM to decide answerability. Experiments over newly constructed datasets show that FUn-FuSIC outperforms suitable adaptations of the SoTA model for KBQA with unanswerability, and the SoTA model for answerable-only few-shot-transfer KBQA.

Related papers

• Crafting Interpretable Embeddings by Asking LLMs Questions [89.49960984640363]
  Large language models (LLMs) have rapidly improved text embeddings for a growing array of natural-language processing tasks. We introduce question-answering embeddings (QA-Emb), embeddings where each feature represents an answer to a yes/no question asked to an LLM. We use QA-Emb to flexibly generate interpretable models for predicting fMRI voxel responses to language stimuli.
  arXiv  Detail & Related papers  (2024-05-26T22:30:29Z)
• RetinaQA: A Robust Knowledge Base Question Answering Model for both Answerable and Unanswerable Questions [23.73807255464977]
  State-of-the-art Knowledge Base Question Answering (KBQA) models assume all questions to be answerable. We propose RetinaQA, a new model that unifies two key ideas in a single KBQA architecture. We show that RetinaQA significantly outperforms adaptations of state-of-the-art KBQA models in handling both answerable and unanswerable questions.
  arXiv  Detail & Related papers  (2024-03-16T08:08:20Z)
• Chain-of-Discussion: A Multi-Model Framework for Complex Evidence-Based Question Answering [55.295699268654545]
  We propose a novel Chain-of-Discussion framework to leverage the synergy among open-source Large Language Models. Our experiments show that discussions among multiple LLMs play a vital role in enhancing the quality of answers.
  arXiv  Detail & Related papers  (2024-02-26T05:31:34Z)
• Few-shot Transfer Learning for Knowledge Base Question Answering: Fusing Supervised Models with In-Context Learning [20.80841972133938]
  Existing Knowledge Base Question Answering (KBQA) architectures are hungry for annotated data. We introduce the problem of few-shot transfer learning for KBQA, where the target domain offers only a few labeled examples. We propose a novel KBQA architecture called FuSIC-KBQA that performs KB-retrieval using multiple source-trained retrievers.
  arXiv  Detail & Related papers  (2023-11-15T11:56:56Z)
• ChatKBQA: A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Models [19.85526116658481]
  We introduce ChatKBQA, a novel and simple generate-then-retrieve KBQA framework. Experimental results show that ChatKBQA achieves new state-of-the-art performance on standard KBQA datasets. This work can also be regarded as a new paradigm for combining LLMs with knowledge graphs for interpretable and knowledge-required question answering.
  arXiv  Detail & Related papers  (2023-10-13T09:45:14Z)
• Open-Set Knowledge-Based Visual Question Answering with Inference Paths [79.55742631375063]
  The purpose of Knowledge-Based Visual Question Answering (KB-VQA) is to provide a correct answer to the question with the aid of external knowledge bases. We propose a new retriever-ranker paradigm of KB-VQA, Graph pATH rankER (GATHER for brevity) Specifically, it contains graph constructing, pruning, and path-level ranking, which not only retrieves accurate answers but also provides inference paths that explain the reasoning process.
  arXiv  Detail & Related papers  (2023-10-12T09:12:50Z)
• FlexKBQA: A Flexible LLM-Powered Framework for Few-Shot Knowledge Base Question Answering [16.88132219032486]
  We introduce FlexKBQA to mitigate the burden associated with manual annotation. We leverage Large Language Models (LLMs) as program translators for addressing the challenges inherent in the few-shot KBQA task. Specifically, FlexKBQA leverages automated algorithms to sample diverse programs, such as SPARQL queries, from the knowledge base. We observe that under the few-shot even the more challenging zero-shot scenarios, FlexKBQA achieves impressive results with a few annotations.
  arXiv  Detail & Related papers  (2023-08-23T11:00:36Z)
• Momentum Contrastive Pre-training for Question Answering [54.57078061878619]
  MCROSS introduces a momentum contrastive learning framework to align the answer probability between cloze-like and natural query-passage sample pairs. Our method achieves noticeable improvement compared with all baselines in both supervised and zero-shot scenarios.
  arXiv  Detail & Related papers  (2022-12-12T08:28:22Z)
• DecAF: Joint Decoding of Answers and Logical Forms for Question Answering over Knowledge Bases [81.19499764899359]
  We propose a novel framework DecAF that jointly generates both logical forms and direct answers. DecAF achieves new state-of-the-art accuracy on WebQSP, FreebaseQA, and GrailQA benchmarks.
  arXiv  Detail & Related papers  (2022-09-30T19:51:52Z)
• Can NLI Models Verify QA Systems' Predictions? [34.46234860404459]
  We explore the use of natural language inference (NLI) to build robust question answering systems. We leverage large pre-trained models and recent prior datasets to construct powerful question converter and decontextualization modules. We show that our NLI approach can generally improve the confidence estimation of a QA model across different domains.
  arXiv  Detail & Related papers  (2021-04-18T06:03:07Z)
• Harvesting and Refining Question-Answer Pairs for Unsupervised QA [95.9105154311491]
  We introduce two approaches to improve unsupervised Question Answering (QA) First, we harvest lexically and syntactically divergent questions from Wikipedia to automatically construct a corpus of question-answer pairs (named as RefQA) Second, we take advantage of the QA model to extract more appropriate answers, which iteratively refines data over RefQA.
  arXiv  Detail & Related papers  (2020-05-06T15:56:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.