Evaluating the Effectiveness of Retrieval-Augmented Large Language Models in Scientific Document Reasoning

• URL: http://arxiv.org/abs/2311.04348v1
• Date: Tue, 7 Nov 2023 21:09:57 GMT
• Title: Evaluating the Effectiveness of Retrieval-Augmented Large Language Models in Scientific Document Reasoning
• Abstract summary: Large Language Model (LLM) often provide seemingly plausible but not factual information, often referred to as hallucinations. Retrieval-augmented LLMs provide a non-parametric approach to solve these issues by retrieving relevant information from external data sources. We critically evaluate these models in their ability to perform in scientific document reasoning tasks.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite the dramatic progress in Large Language Model (LLM) development, LLMs often provide seemingly plausible but not factual information, often referred to as hallucinations. Retrieval-augmented LLMs provide a non-parametric approach to solve these issues by retrieving relevant information from external data sources and augment the training process. These models help to trace evidence from an externally provided knowledge base allowing the model predictions to be better interpreted and verified. In this work, we critically evaluate these models in their ability to perform in scientific document reasoning tasks. To this end, we tuned multiple such model variants with science-focused instructions and evaluated them on a scientific document reasoning benchmark for the usefulness of the retrieved document passages. Our findings suggest that models justify predictions in science tasks with fabricated evidence and leveraging scientific corpus as pretraining data does not alleviate the risk of evidence fabrication.

Related papers

• FALSIFYBENCH: Evaluating Inductive Reasoning in LLMs with Rule Discovery Games [2.959715295394151]
  We introduce FALSIFYBENCH, an evaluation framework for hypothesis-driven reasoning inspired by the classic Wason 2-4-6 task.<n>This task captures key elements of scientific reasoning: hypothesis generation, evidence gathering, and belief revision in response to both confirming and disconfirming evidence.<n>Our evaluation of 12 LLMs across model families and scales shows that reasoning models are generally stronger scientific reasoners than instruction-tuned models, although no model comes close to optimal performance.
  arXiv  Detail & Related papers  (2026-06-03T11:33:17Z)
• Understanding the Role of Hallucination in Reinforcement Post-Training of Multimodal Reasoning Models [54.50728814348712]
  reinforcement learning (RL) has inspired the growing adoption of RL for post-training Multimodal Large Language Models (MLLMs) to enhance their visual reasoning capabilities.<n>We propose the Hallucination-as-Cue Framework to investigate the effects of RL-based post-training on multimodal reasoning models from the perspective of model hallucination.
  arXiv  Detail & Related papers  (2026-04-03T16:56:34Z)
• Leveraging LLM Parametric Knowledge for Fact Checking without Retrieval [60.25608870901428]
  Trustworthiness is a core research challenge for agentic AI systems built on Large Language Models (LLMs)<n>We propose the task of fact-checking without retrieval, focusing on the verification of arbitrary natural language claims, independent of their source robustness.
  arXiv  Detail & Related papers  (2026-03-05T18:42:51Z)
• Evaluating Large Language Models in Scientific Discovery [91.732562776782]
  Large language models (LLMs) are increasingly applied to scientific research, yet prevailing science benchmarks probe decontextualized knowledge.<n>We introduce a scenario-grounded benchmark that evaluates LLMs across biology, chemistry, materials, and physics.<n>The framework assesses models at two levels: (i) question-level accuracy on scenario-tied items and (ii) project-level performance.
  arXiv  Detail & Related papers  (2025-12-17T16:20:03Z)
• Best Practices for Machine Learning Experimentation in Scientific Applications [3.093890460224435]
  This paper presents a practical and structured guide for conducting machine learning experiments in scientific applications.<n>We outline a step-by-step workflow, from dataset preparation to model selection and evaluation.<n>We propose metrics that account for overfitting and instability across folds, including the Logarithmic Overfitting Ratio (LOR) and the Composite Overfitting Score (COS)
  arXiv  Detail & Related papers  (2025-11-26T13:02:31Z)
• Rewriting History: A Recipe for Interventional Analyses to Study Data Effects on Model Behavior [58.58249548116766]
  We present an experimental recipe for studying the relationship between training data and language model (LM) behavior.<n>We outline steps for intervening on data batches and then retraining model checkpoints over that data to test hypotheses relating data to behavior.
  arXiv  Detail & Related papers  (2025-10-16T03:22:48Z)
• Knowledge-Driven Hallucination in Large Language Models: An Empirical Study on Process Modeling [46.05103857535919]
  The utility of Large Language Models in analytical tasks is rooted in their vast pre-trained knowledge.<n>This same capability introduces a critical risk of what we term knowledge-driven hallucination.<n>This paper investigates this phenomenon by evaluating LLMs on the task of automated process modeling.
  arXiv  Detail & Related papers  (2025-09-18T18:27:30Z)
• A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers [251.23085679210206]
  Scientific Large Language Models (Sci-LLMs) are transforming how knowledge is represented, integrated, and applied in scientific research.<n>This survey reframes the development of Sci-LLMs as a co-evolution between models and their underlying data substrate.<n>We formulate a unified taxonomy of scientific data and a hierarchical model of scientific knowledge.
  arXiv  Detail & Related papers  (2025-08-28T18:30:52Z)
• Atomic Reasoning for Scientific Table Claim Verification [83.14588611859826]
  Non-experts are susceptible to misleading claims based on scientific tables due to their high information density and perceived credibility.<n>Existing table claim verification models, including state-of-the-art large language models (LLMs), often struggle with precise fine-grained reasoning.<n>Inspired by Cognitive Load Theory, we propose that enhancing a model's ability to interpret table-based claims involves reducing cognitive load.
  arXiv  Detail & Related papers  (2025-06-08T02:46:22Z)
• FACT-AUDIT: An Adaptive Multi-Agent Framework for Dynamic Fact-Checking Evaluation of Large Language Models [79.41859481668618]
  Large Language Models (LLMs) have significantly advanced the fact-checking studies. Existing automated fact-checking evaluation methods rely on static datasets and classification metrics. We introduce FACT-AUDIT, an agent-driven framework that adaptively and dynamically assesses LLMs' fact-checking capabilities.
  arXiv  Detail & Related papers  (2025-02-25T07:44:22Z)
• SciClaimHunt: A Large Dataset for Evidence-based Scientific Claim Verification [7.421845364041002]
  We introduce two large-scale datasets, SciClaimHunt and SciClaimHunt_Num, derived from scientific research papers. We propose several baseline models tailored for scientific claim verification to assess the effectiveness of these datasets. We evaluate models trained on SciClaimHunt and SciClaimHunt_Num against existing scientific claim verification datasets to gauge their quality and reliability.
  arXiv  Detail & Related papers  (2025-02-14T08:34:26Z)
• A Debate-Driven Experiment on LLM Hallucinations and Accuracy [7.821303946741665]
  This study investigates the phenomenon of hallucination in large language models (LLMs) Multiple instances of GPT-4o-Mini models engage in a debate-like interaction prompted with questions from the TruthfulQA dataset. One model is deliberately instructed to generate plausible but false answers while the other models are asked to respond truthfully.
  arXiv  Detail & Related papers  (2024-10-25T11:41:27Z)
• Forewarned is Forearmed: Leveraging LLMs for Data Synthesis through Failure-Inducing Exploration [90.41908331897639]
  Large language models (LLMs) have significantly benefited from training on diverse, high-quality task-specific data. We present a novel approach, ReverseGen, designed to automatically generate effective training samples.
  arXiv  Detail & Related papers  (2024-10-22T06:43:28Z)
• Reasoning and Tools for Human-Level Forecasting [0.4261908132550109]
  We present Reasoning and Tools for Forecasting (RTF), a framework of reasoning-and-acting (ReAct) agents that can retrieve updated information and run numerical simulation with equipped tools. We evaluate our model with questions from competitive forecasting platforms and demonstrate that our method is competitive with and can outperform human predictions.
  arXiv  Detail & Related papers  (2024-08-21T23:42:06Z)
• Multimodal Misinformation Detection using Large Vision-Language Models [7.505532091249881]
  Large language models (LLMs) have shown remarkable performance in various tasks. Few approaches consider evidence retrieval as part of misinformation detection. We propose a novel re-ranking approach for multimodal evidence retrieval.
  arXiv  Detail & Related papers  (2024-07-19T13:57:11Z)
• Extracting Training Data from Unconditional Diffusion Models [76.85077961718875]
  diffusion probabilistic models (DPMs) are being employed as mainstream models for generative artificial intelligence (AI) We aim to establish a theoretical understanding of memorization in DPMs with 1) a memorization metric for theoretical analysis, 2) an analysis of conditional memorization with informative and random labels, and 3) two better evaluation metrics for measuring memorization. Based on the theoretical analysis, we propose a novel data extraction method called textbfSurrogate condItional Data Extraction (SIDE) that leverages a trained on generated data as a surrogate condition to extract training data directly from unconditional diffusion models.
  arXiv  Detail & Related papers  (2024-06-18T16:20:12Z)
• Low-rank finetuning for LLMs: A fairness perspective [54.13240282850982]
  Low-rank approximation techniques have become the de facto standard for fine-tuning Large Language Models. This paper investigates the effectiveness of these methods in capturing the shift of fine-tuning datasets from the initial pre-trained data distribution. We show that low-rank fine-tuning inadvertently preserves undesirable biases and toxic behaviors.
  arXiv  Detail & Related papers  (2024-05-28T20:43:53Z)
• Empirical evaluation of Uncertainty Quantification in Retrieval-Augmented Language Models for Science [0.0]
  This study investigates how uncertainty scores vary when scientific knowledge is incorporated as pretraining and retrieval data. We observe that an existing RALM finetuned with scientific knowledge as the retrieval data tends to be more confident in generating predictions. We also found that RALMs are overconfident in their predictions, making inaccurate predictions more confidently than accurate ones.
  arXiv  Detail & Related papers  (2023-11-15T20:42:11Z)
• Interpretable Medical Diagnostics with Structured Data Extraction by Large Language Models [59.89454513692417]
  Tabular data is often hidden in text, particularly in medical diagnostic reports. We propose a novel, simple, and effective methodology for extracting structured tabular data from textual medical reports, called TEMED-LLM. We demonstrate that our approach significantly outperforms state-of-the-art text classification models in medical diagnostics.
  arXiv  Detail & Related papers  (2023-06-08T09:12:28Z)
• Large Language Models with Controllable Working Memory [64.71038763708161]
  Large language models (LLMs) have led to a series of breakthroughs in natural language processing (NLP) What further sets these models apart is the massive amounts of world knowledge they internalize during pretraining. How the model's world knowledge interacts with the factual information presented in the context remains under explored.
  arXiv  Detail & Related papers  (2022-11-09T18:58:29Z)
• Measuring Causal Effects of Data Statistics on Language Model's `Factual' Predictions [59.284907093349425]
  Large amounts of training data are one of the major reasons for the high performance of state-of-the-art NLP models. We provide a language for describing how training data influences predictions, through a causal framework. Our framework bypasses the need to retrain expensive models and allows us to estimate causal effects based on observational data alone.
  arXiv  Detail & Related papers  (2022-07-28T17:36:24Z)
• Scientific Inference With Interpretable Machine Learning: Analyzing Models to Learn About Real-World Phenomena [4.312340306206884]
  Interpretable machine learning offers a solution by analyzing models holistically to derive interpretations. Current IML research is focused on auditing ML models rather than leveraging them for scientific inference. We present a framework for designing IML methods-termed 'property descriptors' that illuminate not just the model, but also the phenomenon it represents.
  arXiv  Detail & Related papers  (2022-06-11T10:13:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.