KBQA-o1: Agentic Knowledge Base Question Answering with Monte Carlo Tree Search

• URL: http://arxiv.org/abs/2501.18922v1
• Date: Fri, 31 Jan 2025 06:59:49 GMT
• Title: KBQA-o1: Agentic Knowledge Base Question Answering with Monte Carlo Tree Search
• Authors: Haoran Luo, Haihong E, Yikai Guo, Qika Lin, Xiaobao Wu, Xinyu Mu, Wenhao Liu, Meina Song, Yifan Zhu, Luu Anh Tuan,
• Abstract summary: Knowledge Base Question Answering (KBQA) aims to answer natural language questions with a large-scale structured knowledge base (KB) Despite advancements with large language models (LLMs), KBQA still faces challenges in weak KB awareness, imbalance between effectiveness and efficiency, and high reliance on annotated data. We propose KBQA-o1, a novel agentic KBQA method with Monte Carlo Tree Search (MCTS) Experimental results show that KBQA-o1 outperforms previous low-resource KBQA methods with limited annotated data.
• Score: 30.901330193491457
• License:
• Abstract: Knowledge Base Question Answering (KBQA) aims to answer natural language questions with a large-scale structured knowledge base (KB). Despite advancements with large language models (LLMs), KBQA still faces challenges in weak KB awareness, imbalance between effectiveness and efficiency, and high reliance on annotated data. To address these challenges, we propose KBQA-o1, a novel agentic KBQA method with Monte Carlo Tree Search (MCTS). It introduces a ReAct-based agent process for stepwise logical form generation with KB environment exploration. Moreover, it employs MCTS, a heuristic search method driven by policy and reward models, to balance agentic exploration's performance and search space. With heuristic exploration, KBQA-o1 generates high-quality annotations for further improvement by incremental fine-tuning. Experimental results show that KBQA-o1 outperforms previous low-resource KBQA methods with limited annotated data, boosting Llama-3.1-8B model's GrailQA F1 performance to 78.5% compared to 48.5% of the previous sota method with GPT-3.5-turbo.

Related papers

• Beyond Seen Data: Improving KBQA Generalization Through Schema-Guided Logical Form Generation [36.01290160488469]
  Knowledge base question answering (KBQA) aims to answer user questions in natural language using rich human knowledge stored in large KBs. Current KBQA methods struggle with unseen knowledge base elements at test time. We introduce SG-KBQA: a novel model that injects schema contexts into entity retrieval and logical form generation.
  arXiv  Detail & Related papers  (2025-02-18T10:53:41Z)
• A Learn-Then-Reason Model Towards Generalization in Knowledge Base Question Answering [17.281005999581865]
  Large-scale knowledge bases (KBs) like Freebase and Wikidata house millions of structured knowledge. Knowledge Base Question Answering (KBQA) provides a user-friendly way to access these valuable KBs via asking natural language questions. This paper develops KBLLaMA, which follows a learn-then-reason framework to inject new KB knowledge into a large language model for flexible end-to-end KBQA.
  arXiv  Detail & Related papers  (2024-06-20T22:22:41Z)
• ChatKBQA: A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Models [19.85526116658481]
  We introduce ChatKBQA, a novel and simple generate-then-retrieve KBQA framework. Experimental results show that ChatKBQA achieves new state-of-the-art performance on standard KBQA datasets. This work can also be regarded as a new paradigm for combining LLMs with knowledge graphs for interpretable and knowledge-required question answering.
  arXiv  Detail & Related papers  (2023-10-13T09:45:14Z)
• FC-KBQA: A Fine-to-Coarse Composition Framework for Knowledge Base Question Answering [24.394908238940904]
  We propose a Fine-to-Coarse Composition framework for KBQA (FC-KBQA) to ensure the generalization ability and executability of the logical expression. FC-KBQA derives new state-of-the-art performance on GrailQA and WebQSP, and runs 4 times faster than the baseline.
  arXiv  Detail & Related papers  (2023-06-26T14:19:46Z)
• SYGMA: System for Generalizable Modular Question Answering OverKnowledge Bases [57.89642289610301]
  We present SYGMA, a modular approach facilitating general-izability across multiple knowledge bases and multiple rea-soning types. We demonstrate effectiveness of our system by evaluating on datasets belonging to two distinct knowledge bases,DBpedia and Wikidata.
  arXiv  Detail & Related papers  (2021-09-28T01:57:56Z)
• RnG-KBQA: Generation Augmented Iterative Ranking for Knowledge Base Question Answering [57.94658176442027]
  We present RnG-KBQA, a Rank-and-Generate approach for KBQA. We achieve new state-of-the-art results on GrailQA and WebQSP datasets.
  arXiv  Detail & Related papers  (2021-09-17T17:58:28Z)
• Beyond I.I.D.: Three Levels of Generalization for Question Answering on Knowledge Bases [63.43418760818188]
  We release a new large-scale, high-quality dataset with 64,331 questions, GrailQA. We propose a novel BERT-based KBQA model. The combination of our dataset and model enables us to thoroughly examine and demonstrate, for the first time, the key role of pre-trained contextual embeddings like BERT in the generalization of KBQA.
  arXiv  Detail & Related papers  (2020-11-16T06:36:26Z)
• Learning Knowledge Bases with Parameters for Task-Oriented Dialogue Systems [79.02430277138801]
  The knowledge base (KB) plays an essential role in fulfilling user requests. End-to-end systems use the KB directly as input, but they cannot scale when the KB is larger than a few hundred entries. We propose a method to embed the KB, of any size, directly into the model parameters.
  arXiv  Detail & Related papers  (2020-09-28T22:13:54Z)
• A Survey on Complex Question Answering over Knowledge Base: Recent Advances and Challenges [71.4531144086568]
  Question Answering (QA) over Knowledge Base (KB) aims to automatically answer natural language questions. Researchers have shifted their attention from simple questions to complex questions, which require more KB triples and constraint inference.
  arXiv  Detail & Related papers  (2020-07-26T07:13:32Z)
• Faithful Embeddings for Knowledge Base Queries [97.5904298152163]
  deductive closure of an ideal knowledge base (KB) contains exactly the logical queries that the KB can answer. In practice KBs are both incomplete and over-specified, failing to answer some queries that have real-world answers. We show that inserting this new QE module into a neural question-answering system leads to substantial improvements over the state-of-the-art.
  arXiv  Detail & Related papers  (2020-04-07T19:25:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.