Question-Answer Extraction from Scientific Articles Using Knowledge Graphs and Large Language Models

• URL: http://arxiv.org/abs/2507.13827v1
• Date: Fri, 18 Jul 2025 11:31:52 GMT
• Title: Question-Answer Extraction from Scientific Articles Using Knowledge Graphs and Large Language Models
• Authors: Hosein Azarbonyad, Zi Long Zhu, Georgios Cheirmpos, Zubair Afzal, Vikrant Yadav, Georgios Tsatsaronis,
• Abstract summary: We propose two distinct approaches for generating Question and Answer pairs from scientific articles.<n>The first approach involves selecting salient paragraphs, using a Large Language Model (LLM) to generate questions.<n>The second approach leverages a Knowledge Graph (KG) for QA generation.
• Score: 1.8637078358591848
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: When deciding to read an article or incorporate it into their research, scholars often seek to quickly identify and understand its main ideas. In this paper, we aim to extract these key concepts and contributions from scientific articles in the form of Question and Answer (QA) pairs. We propose two distinct approaches for generating QAs. The first approach involves selecting salient paragraphs, using a Large Language Model (LLM) to generate questions, ranking these questions by the likelihood of obtaining meaningful answers, and subsequently generating answers. This method relies exclusively on the content of the articles. However, assessing an article's novelty typically requires comparison with the existing literature. Therefore, our second approach leverages a Knowledge Graph (KG) for QA generation. We construct a KG by fine-tuning an Entity Relationship (ER) extraction model on scientific articles and using it to build the graph. We then employ a salient triplet extraction method to select the most pertinent ERs per article, utilizing metrics such as the centrality of entities based on a triplet TF-IDF-like measure. This measure assesses the saliency of a triplet based on its importance within the article compared to its prevalence in the literature. For evaluation, we generate QAs using both approaches and have them assessed by Subject Matter Experts (SMEs) through a set of predefined metrics to evaluate the quality of both questions and answers. Our evaluations demonstrate that the KG-based approach effectively captures the main ideas discussed in the articles. Furthermore, our findings indicate that fine-tuning the ER extraction model on our scientific corpus is crucial for extracting high-quality triplets from such documents.

Related papers

• PeerQA: A Scientific Question Answering Dataset from Peer Reviews [51.95579001315713]
  We present PeerQA, a real-world, scientific, document-level Question Answering dataset.<n>The dataset contains 579 QA pairs from 208 academic articles, with a majority from ML and NLP.<n>We provide a detailed analysis of the collected dataset and conduct experiments establishing baseline systems for all three tasks.
  arXiv  Detail & Related papers  (2025-02-19T12:24:46Z)
• Pointwise Mutual Information as a Performance Gauge for Retrieval-Augmented Generation [78.28197013467157]
  We show that the pointwise mutual information between a context and a question is an effective gauge for language model performance.<n>We propose two methods that use the pointwise mutual information between a document and a question as a gauge for selecting and constructing prompts that lead to better performance.
  arXiv  Detail & Related papers  (2024-11-12T13:14:09Z)
• CiteFusion: An Ensemble Framework for Citation Intent Classification Harnessing Dual-Model Binary Couples and SHAP Analyses [1.7812428873698407]
  CiteFusion addresses the multi-class Citation Intent Classification task on two benchmark datasets: SciCite and ACL-ARC.<n>The framework employs a one-vs-all decomposition of the multi-class task into class-specific binary sub-tasks.<n>Results show that CiteFusion achieves state-of-the-art performance, with Macro-F1 scores of 89.60% on SciCite, and 76.24% on ACL-ARC.
  arXiv  Detail & Related papers  (2024-07-18T09:29:33Z)
• EWEK-QA: Enhanced Web and Efficient Knowledge Graph Retrieval for Citation-based Question Answering Systems [103.91826112815384]
  citation-based QA systems are suffering from two shortcomings. They usually rely only on web as a source of extracted knowledge and adding other external knowledge sources can hamper the efficiency of the system. We propose our enhanced web and efficient knowledge graph (KG) retrieval solution (EWEK-QA) to enrich the content of the extracted knowledge fed to the system.
  arXiv  Detail & Related papers  (2024-06-14T19:40:38Z)
• RelevAI-Reviewer: A Benchmark on AI Reviewers for Survey Paper Relevance [0.8089605035945486]
  We propose RelevAI-Reviewer, an automatic system that conceptualizes the task of survey paper review as a classification problem. We introduce a novel dataset comprised of 25,164 instances. Each instance contains one prompt and four candidate papers, each varying in relevance to the prompt. We develop a machine learning (ML) model capable of determining the relevance of each paper and identifying the most pertinent one.
  arXiv  Detail & Related papers  (2024-06-13T06:42:32Z)
• A Literature Review of Literature Reviews in Pattern Analysis and Machine Intelligence [55.33653554387953]
  Pattern Analysis and Machine Intelligence (PAMI) has led to numerous literature reviews aimed at collecting and fragmented information.<n>This paper presents a thorough analysis of these literature reviews within the PAMI field.<n>We try to address three core research questions: (1) What are the prevalent structural and statistical characteristics of PAMI literature reviews; (2) What strategies can researchers employ to efficiently navigate the growing corpus of reviews; and (3) What are the advantages and limitations of AI-generated reviews compared to human-authored ones.
  arXiv  Detail & Related papers  (2024-02-20T11:28:50Z)
• Graph Reasoning for Question Answering with Triplet Retrieval [33.454090126152714]
  We propose a simple yet effective method to retrieve the most relevant triplets from knowledge graphs (KGs) Our method can outperform state-of-the-art up to 4.6% absolute accuracy.
  arXiv  Detail & Related papers  (2023-05-30T04:46:28Z)
• Evaluating BERT-based Scientific Relation Classifiers for Scholarly Knowledge Graph Construction on Digital Library Collections [5.8962650619804755]
  Inferring semantic relations between related scientific concepts is a crucial step. BERT-based pre-trained models have been popularly explored for automatic relation classification. Existing methods are primarily evaluated on clean texts. To address these limitations, we started by creating OCR-noisy texts.
  arXiv  Detail & Related papers  (2023-05-03T17:32:16Z)
• UniKGQA: Unified Retrieval and Reasoning for Solving Multi-hop Question Answering Over Knowledge Graph [89.98762327725112]
  Multi-hop Question Answering over Knowledge Graph(KGQA) aims to find the answer entities that are multiple hops away from the topic entities mentioned in a natural language question. We propose UniKGQA, a novel approach for multi-hop KGQA task, by unifying retrieval and reasoning in both model architecture and parameter learning.
  arXiv  Detail & Related papers  (2022-12-02T04:08:09Z)
• A Data-driven Latent Semantic Analysis for Automatic Text Summarization using LDA Topic Modelling [0.0]
  This study presents the Latent Dirichlet Allocation (LDA) approach used to perform topic modelling. The visualisation provides an overarching view of the main topics while allowing and attributing deep meaning to the prevalence individual topic. The results suggest the terms ranked purely by considering their probability of the topic prevalence within the processed document.
  arXiv  Detail & Related papers  (2022-07-23T11:04:03Z)
• Science Checker: Extractive-Boolean Question Answering For Scientific Fact Checking [0.0]
  We propose a multi-task approach for verifying the scientific questions based on a joint reasoning from facts and evidence in research articles. With our light and fast proposed architecture, we achieved an average error rate of 4% and a F1-score of 95.6%.
  arXiv  Detail & Related papers  (2022-04-26T12:35:23Z)
• What's New? Summarizing Contributions in Scientific Literature [85.95906677964815]
  We introduce a new task of disentangled paper summarization, which seeks to generate separate summaries for the paper contributions and the context of the work. We extend the S2ORC corpus of academic articles by adding disentangled "contribution" and "context" reference labels. We propose a comprehensive automatic evaluation protocol which reports the relevance, novelty, and disentanglement of generated outputs.
  arXiv  Detail & Related papers  (2020-11-06T02:23:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.