BioGraphletQA: Knowledge-Anchored Generation of Complex QA Datasets

• URL: http://arxiv.org/abs/2604.26048v1
• Date: Tue, 28 Apr 2026 18:33:21 GMT
• Title: BioGraphletQA: Knowledge-Anchored Generation of Complex QA Datasets
• Abstract summary: This paper presents a principled framework for generating complex Question Answering (QA) data.<n>In the core of this framework is a graphlet-anchored generation process, where small subgraphs from a Knowledge Graph (KG) are used in a structured prompt.<n>The first instantiation of this framework is BioGraphletQA, a new biomedical KGQA dataset of 119,856 QA pairs.
• Score: 0.3058685580689604
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents a principled and scalable framework for systematically generating complex Question Answering (QA) data. In the core of this framework is a graphlet-anchored generation process, where small subgraphs from a Knowledge Graph (KG) are used in a structured prompt to control the complexity and ensure the factual grounding of questions generated by Large Language Models. The first instantiation of this framework is BioGraphletQA, a new biomedical KGQA dataset of 119,856 QA pairs. Each entry is grounded in a graphlet of up to five nodes from the OREGANO KG, with most of the pairs being enriched with relevant document snippets from PubMed. We start by demonstrating the framework's value and the dataset's quality through evaluation by a domain expert on 106 QA pairs, confirming the high scientific validity and complexity of the generated data. Secondly, we establish its practical utility by showing that augmenting downstream benchmarks with our data improves accuracy on PubMedQA from 49.2% to 68.5% in a low-resource setting, and on MedQA from a 41.4% baseline to 44.8% in a full-resource setting. Our framework provides a robust and generalizable solution for creating critical resources to advance complex QA tasks, including MCQA and KGQA. All resources supporting this work, including the dataset (https://zenodo.org/records/17381119) and framework code (https://github.com/ieeta-pt/BioGraphletQA), are publicly available to facilitate use, reproducibility and extension.

Related papers

• HOME-KGQA: A Benchmark Dataset for Multimodal Knowledge Graph Question Answering on Household Daily Activities [11.096574335599092]
  We introduce HOME-KGQA, a novel KGQA benchmark built on a multimodal KG of daily household activities.<n>Home-KGQA consists of complex, multi-hop natural language questions paired with graph database query languages.<n>Results show that the LLM-based KGQA methods fail to achieve performance comparable to that on existing datasets.
  arXiv  Detail & Related papers  (2026-05-10T06:00:29Z)
• Open Biomedical Knowledge Graphs at Scale: Construction, Federation, and AI Agent Access with Samyama Graph Database [0.0]
  We present three open-source biomedical knowledge graphs -- Pathways KG, Clinical Trials KG, and Drug Interactions KG.<n>Our contributions are threefold. First, we describe a high-performance graph for constructing large-scale KGs from heterogeneous public data sources.<n>Second, we demonstrate cross-KG federation: loading all three snapshots into a single graph tenant enables property-based joins.
  arXiv  Detail & Related papers  (2026-03-16T10:36:13Z)
• KG20C & KG20C-QA: Scholarly Knowledge Graph Benchmarks for Link Prediction and Question Answering [3.8315541579168353]
  KG20C is a high-quality scholarly knowledge graph constructed from the Microsoft Academic Graph.<n> KG20C-QA is built upon KG20C to support QA tasks on scholarly data.
  arXiv  Detail & Related papers  (2025-12-25T22:29:54Z)
• OIDA-QA: A Multimodal Benchmark for Analyzing the Opioid Industry Documents Archive [50.468138755368805]
  Opioid crisis represents a significant moment in public health.<n>Data and documents disclosed in the UCSF-JHU Opioid Industry Documents Archive (OIDA)<n>In this paper, we tackle this challenge by organizing the original dataset according to document attributes.
  arXiv  Detail & Related papers  (2025-11-13T03:27:32Z)
• Ground-Truth Subgraphs for Better Training and Evaluation of Knowledge Graph Augmented LLMs [3.222543736797976]
  SynthKGQA is a framework for generating high-quality synthetic Knowledge Graph Question Answering datasets from any Knowledge Graph.<n>We show how, in addition to enabling more informative benchmarking of KG retrievers, the data produced with SynthKGQA also allows us to train better models.
  arXiv  Detail & Related papers  (2025-11-06T15:45:18Z)
• Can Knowledge-Graph-based Retrieval Augmented Generation Really Retrieve What You Need? [57.28763506780752]
  GraphFlow is a framework that efficiently retrieves accurate and diverse knowledge required for real-world queries from text-rich KGs.<n>It outperforms strong KG-RAG baselines, including GPT-4o, by 10% on average in hit rate and recall.<n>It also shows strong generalization to unseen KGs, demonstrating its effectiveness and robustness.
  arXiv  Detail & Related papers  (2025-10-18T17:06:49Z)
• Diagnosing and Addressing Pitfalls in KG-RAG Datasets: Toward More Reliable Benchmarking [63.84117489519164]
  Knowledge Graph Question Answering systems rely on high-quality benchmarks to evaluate complex multi-hop reasoning.<n>Despite their widespread use, popular datasets such as WebQSP and CWQ suffer from critical quality issues.<n>We introduce KGQAGen, an LLM-in-the-loop framework that systematically resolves these pitfalls.<n>Our findings advocate for more rigorous benchmark construction and position KGQAGen as a scalable framework for advancing KGQA evaluation.
  arXiv  Detail & Related papers  (2025-05-29T14:44:52Z)
• Ontology-grounded Automatic Knowledge Graph Construction by LLM under Wikidata schema [60.42231674887294]
  We propose an ontology-grounded approach to Knowledge Graph (KG) construction using Large Language Models (LLMs) on a knowledge base.<n>We ground generation of KG with the authored ontology based on extracted relations to ensure consistency and interpretability.<n>Our work presents a promising direction for scalable KG construction pipeline with minimal human intervention, that yields high quality and human-interpretable KGs.
  arXiv  Detail & Related papers  (2024-12-30T13:36:05Z)
• Hybrid-SQuAD: Hybrid Scholarly Question Answering Dataset [8.867885891794877]
  We introduce Hybrid-SQuAD, a novel large-scale Scholarly Question Answering dataset.<n>The dataset consists of 10.5K question-answer pairs generated by a large language model.<n>We propose a RAG-based baseline hybrid QA model, achieving an exact match score of 69.65 on the Hybrid-SQuAD test set.
  arXiv  Detail & Related papers  (2024-12-03T19:37:00Z)
• KET-QA: A Dataset for Knowledge Enhanced Table Question Answering [63.56707527868466]
  We propose to use a knowledge base (KB) as the external knowledge source for TableQA. Every question requires the integration of information from both the table and the sub-graph to be answered. We design a retriever-reasoner structured pipeline model to extract pertinent information from the vast knowledge sub-graph.
  arXiv  Detail & Related papers  (2024-05-13T18:26:32Z)
• Knowledge Graph Question Answering for Materials Science (KGQA4MAT): Developing Natural Language Interface for Metal-Organic Frameworks Knowledge Graph (MOF-KG) Using LLM [35.208135795371795]
  We present a benchmark dataset for Knowledge Graph Question Answering in Materials Science (KGQA4MAT) A knowledge graph for metal-organic frameworks (MOF-KG) has been constructed by integrating structured databases and knowledge extracted from the literature. We have developed a benchmark comprised of 161 complex questions involving comparison, aggregation, and complicated graph structures.
  arXiv  Detail & Related papers  (2023-09-20T14:43:43Z)
• Knowledge Graph Question Answering Leaderboard: A Community Resource to Prevent a Replication Crisis [61.740077541531726]
  We provide a new central and open leaderboard for any KGQA benchmark dataset as a focal point for the community. Our analysis highlights existing problems during the evaluation of KGQA systems.
  arXiv  Detail & Related papers  (2022-01-20T13:46:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.