Enhancing Scientific Literature Chatbots with Retrieval-Augmented Generation: A Performance Evaluation of Vector and Graph-Based Systems

• URL: http://arxiv.org/abs/2602.17856v1
• Date: Thu, 19 Feb 2026 21:42:02 GMT
• Title: Enhancing Scientific Literature Chatbots with Retrieval-Augmented Generation: A Performance Evaluation of Vector and Graph-Based Systems
• Authors: Hamideh Ghanadian, Amin Kamali, Mohammad Hossein Tekieh,
• Abstract summary: This paper investigates the enhancement of scientific literature through retrieval-augmented generation (RAG)<n>The proposed chatbots leverages both structured (graph) and unstructured (vector) databases to access scientific articles and gray literature.<n> Benchmark test sets were generated using a GPT model, with selected outputs annotated for evaluation.
• Score: 1.0832844764942349
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper investigates the enhancement of scientific literature chatbots through retrieval-augmented generation (RAG), with a focus on evaluating vector- and graph-based retrieval systems. The proposed chatbot leverages both structured (graph) and unstructured (vector) databases to access scientific articles and gray literature, enabling efficient triage of sources according to research objectives. To systematically assess performance, we examine two use-case scenarios: retrieval from a single uploaded document and retrieval from a large-scale corpus. Benchmark test sets were generated using a GPT model, with selected outputs annotated for evaluation. The comparative analysis emphasizes retrieval accuracy and response relevance, providing insight into the strengths and limitations of each approach. The findings demonstrate the potential of hybrid RAG systems to improve accessibility to scientific knowledge and to support evidence-based decision making.

Related papers

• Automated Research Article Classification and Recommendation Using NLP and ML [0.5486463492959637]
  This paper presents an automated framework for research article classification and recommendation.<n>We use a large-scale arXiv.org dataset spanning more than three decades.<n>To complement classification, we incorporate a recommendation module based on the cosine similarity of vectorized articles.
  arXiv  Detail & Related papers  (2025-10-07T01:24:35Z)
• Biomedical Literature Q&A System Using Retrieval-Augmented Generation (RAG) [0.0]
  This work presents a Biomedical Literature Question Answering (Q&A) system based on a Retrieval-Augmented Generation architecture.<n>The system integrates diverse sources, including PubMed articles, curated Q&A datasets, and medical encyclopedias.<n>The system supports both general medical queries and domain-specific tasks, with a focused evaluation on breast cancer literature.
  arXiv  Detail & Related papers  (2025-09-05T21:29:52Z)
• HySemRAG: A Hybrid Semantic Retrieval-Augmented Generation Framework for Automated Literature Synthesis and Methodological Gap Analysis [55.2480439325792]
  HySemRAG is a framework that combines Extract, Transform, Load (ETL) pipelines with Retrieval-Augmented Generation (RAG)<n>System addresses limitations in existing RAG architectures through a multi-layered approach.
  arXiv  Detail & Related papers  (2025-08-01T20:30:42Z)
• Controlled Retrieval-augmented Context Evaluation for Long-form RAG [58.14561461943611]
  Retrieval-augmented generation (RAG) enhances large language models by incorporating context retrieved from external knowledge sources.<n>We argue that providing a comprehensive retrieval-augmented context is important for long-form RAG tasks like report generation.<n>We introduce CRUX, a framework designed to directly assess retrieval-augmented contexts.
  arXiv  Detail & Related papers  (2025-06-24T23:17:48Z)
• Optimizing Retrieval-Augmented Generation of Medical Content for Spaced Repetition Learning [0.46603287532620746]
  This paper presents a pipeline employing comments generation for Poland's State Retrieval Examination (PES) based on verified resources.<n>The system integrates these generated comments and source documents with a spaced repetition learning algorithm to enhance knowledge retention.<n> Rigorous evaluation by medical annotators demonstrates improvements in key metrics such as document relevance, credibility, and logical coherence of generated content.
  arXiv  Detail & Related papers  (2025-02-23T20:56:31Z)
• CG-RAG: Research Question Answering by Citation Graph Retrieval-Augmented LLMs [9.718354494802002]
  Contextualized Graph Retrieval-Augmented Generation (CG-RAG) is a novel framework that integrates sparse and dense retrieval signals within graph structures.<n>First, we propose a contextual graph representation for citation graphs, effectively capturing both explicit and implicit connections within and across documents.<n>Second, we introduce Lexical-Semantic Graph Retrieval (LeSeGR), which seamlessly integrates sparse and dense retrieval signals with graph encoding.<n>Third, we present a context-aware generation strategy that utilizes the retrieved graph-structured information to generate precise and contextually enriched responses.
  arXiv  Detail & Related papers  (2025-01-25T04:18:08Z)
• Are Large Language Models Good Classifiers? A Study on Edit Intent Classification in Scientific Document Revisions [62.12545440385489]
  Large language models (LLMs) have brought substantial advancements in text generation, but their potential for enhancing classification tasks remains underexplored. We propose a framework for thoroughly investigating fine-tuning LLMs for classification, including both generation- and encoding-based approaches. We instantiate this framework in edit intent classification (EIC), a challenging and underexplored classification task.
  arXiv  Detail & Related papers  (2024-10-02T20:48:28Z)
• Exploring Information Retrieval Landscapes: An Investigation of a Novel Evaluation Techniques and Comparative Document Splitting Methods [0.0]
  In this study, the structured nature of textbooks, the conciseness of articles, and the narrative complexity of novels are shown to require distinct retrieval strategies. A novel evaluation technique is introduced, utilizing an open-source model to generate a comprehensive dataset of question-and-answer pairs. The evaluation employs weighted scoring metrics, including SequenceMatcher, BLEU, METEOR, and BERT Score, to assess the system's accuracy and relevance.
  arXiv  Detail & Related papers  (2024-09-13T02:08:47Z)
• SeRTS: Self-Rewarding Tree Search for Biomedical Retrieval-Augmented Generation [50.26966969163348]
  Large Language Models (LLMs) have shown great potential in the biomedical domain with the advancement of retrieval-augmented generation (RAG) Existing retrieval-augmented approaches face challenges in addressing diverse queries and documents, particularly for medical knowledge queries. We propose Self-Rewarding Tree Search (SeRTS) based on Monte Carlo Tree Search (MCTS) and a self-rewarding paradigm.
  arXiv  Detail & Related papers  (2024-06-17T06:48:31Z)
• A Literature Review of Literature Reviews in Pattern Analysis and Machine Intelligence [51.26815896167173]
  We present a comprehensive tertiary analysis of PAMI reviews along three complementary dimensions.<n>Our analyses reveal distinctive organizational patterns as well as persistent gaps in current review practices.<n>Finally, our evaluation of state-of-the-art AI-generated reviews indicates encouraging advances in coherence and organization.
  arXiv  Detail & Related papers  (2024-02-20T11:28:50Z)
• Evaluating Generative Ad Hoc Information Retrieval [58.800799175084286]
  generative retrieval systems often directly return a grounded generated text as a response to a query. Quantifying the utility of the textual responses is essential for appropriately evaluating such generative ad hoc retrieval.
  arXiv  Detail & Related papers  (2023-11-08T14:05:00Z)
• Opportunities and Challenges of Deep Learning Methods for Electrocardiogram Data: A Systematic Review [62.490310870300746]
  The electrocardiogram (ECG) is one of the most commonly used diagnostic tools in medicine and healthcare. Deep learning methods have achieved promising results on predictive healthcare tasks using ECG signals. This paper presents a systematic review of deep learning methods for ECG data from both modeling and application perspectives.
  arXiv  Detail & Related papers  (2019-12-28T02:44:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.