Evaluation of Causal Reasoning for Large Language Models in Contextualized Clinical Scenarios of Laboratory Test Interpretation

• URL: http://arxiv.org/abs/2509.16372v1
• Date: Fri, 19 Sep 2025 19:28:12 GMT
• Title: Evaluation of Causal Reasoning for Large Language Models in Contextualized Clinical Scenarios of Laboratory Test Interpretation
• Abstract summary: This study evaluates causal reasoning in large language models (LLMs) using 99 clinically grounded laboratory test scenarios.<n>We examined common laboratory tests such as hemoglobin A1c, creatinine, and vitamin D, and paired them with relevant causal factors including age, gender, obesity, and smoking.<n>Two models - GPT-o1 and Llama-3.2-8b-instruct - were tested, with responses evaluated by four medically trained human experts.
• Score: 1.3321309556812102
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This study evaluates causal reasoning in large language models (LLMs) using 99 clinically grounded laboratory test scenarios aligned with Pearl's Ladder of Causation: association, intervention, and counterfactual reasoning. We examined common laboratory tests such as hemoglobin A1c, creatinine, and vitamin D, and paired them with relevant causal factors including age, gender, obesity, and smoking. Two LLMs - GPT-o1 and Llama-3.2-8b-instruct - were tested, with responses evaluated by four medically trained human experts. GPT-o1 demonstrated stronger discriminative performance (AUROC overall = 0.80 +/- 0.12) compared to Llama-3.2-8b-instruct (0.73 +/- 0.15), with higher scores across association (0.75 vs 0.72), intervention (0.84 vs 0.70), and counterfactual reasoning (0.84 vs 0.69). Sensitivity (0.90 vs 0.84) and specificity (0.93 vs 0.80) were also greater for GPT-o1, with reasoning ratings showing similar trends. Both models performed best on intervention questions and worst on counterfactuals, particularly in altered outcome scenarios. These findings suggest GPT-o1 provides more consistent causal reasoning, but refinement is required before adoption in high-stakes clinical applications.

Related papers

• CaresAI at SMM4H-HeaRD 2026: Predicting TNM Staging [2.7998963147546143]
  This study aims to predict Tumor, Node, and Metastasis (TNM) stage labels independently, with the Cancer Genome Atlas (TCGA) pathology report as the sixth shared task of SMM4H-HeaRD 2026.<n>We explore both classical and deep learning approaches using Term Frequency-Inverse Document Frequency (TF-IDF) features and embeddings from ClinicalBERT, BioBERT, and PubMedBERT.<n>Our results show that individual embeddings perform similarly to the TNM label classification, while their combination improves its predictive ability.
  arXiv  Detail & Related papers  (2026-07-03T16:25:35Z)
• Dialysis Risk Prediction and Treatment Effect Estimation for AKI patients using Longitudinal Electronic Health Records [5.2513348694877156]
  We constructed a fixed-window EHR cohort and modeled sequences of diagnoses, procedures, and medications with kidney laboratory trends.<n>A transformer-based causal multi-head model was trained to estimate drug- and ingredient-level average treatment effects.<n>Results showed partial protective-direction support for ACE/ARB exposures and worsening-direction signals for loop diuretics.
  arXiv  Detail & Related papers  (2026-04-27T14:41:37Z)
• PanCanBench: A Comprehensive Benchmark for Evaluating Large Language Models in Pancreatic Oncology [48.732366302949515]
  Large language models (LLMs) have achieved expert-level performance on standardized examinations, yet multiple-choice accuracy poorly reflects real-world clinical utility and safety.<n>We developed a human-in-the-loop pipeline to create expert rubrics for de-identified patient questions.<n>We evaluated 22 proprietary and open-source LLMs using an LLM-as-a-judge framework, measuring clinical completeness, factual accuracy, and web-search integration.
  arXiv  Detail & Related papers  (2026-03-02T00:50:39Z)
• Robust Multicentre Detection and Classification of Colorectal Liver Metastases on CT: Application of Foundation Models [11.274035647041762]
  We developed a foundation model-based AI pipeline for patient-level classification and lesion-level detection of CRLM on CT.<n> UMedPT achieved the best performance and was fine-tuned with a head for classification and an FCOS-based head for lesion detection.
  arXiv  Detail & Related papers  (2026-01-12T14:35:29Z)
• Explainable Admission-Level Predictive Modeling for Prolonged Hospital Stay in Elderly Populations: Challenges in Low- and Middle-Income Countries [65.4286079244589]
  Prolonged length of stay (pLoS) is a significant factor associated with the risk of adverse in-hospital events.<n>We develop and explain a predictive model for pLos using admission-level patient and hospital administrative data.
  arXiv  Detail & Related papers  (2026-01-07T23:35:24Z)
• A Novel Attention-Augmented Wavelet YOLO System for Real-time Brain Vessel Segmentation on Transcranial Color-coded Doppler [49.03919553747297]
  We propose an AI-powered, real-time CoW auto-segmentation system capable of efficiently capturing cerebral arteries.<n>No prior studies have explored AI-driven cerebrovascular segmentation using Transcranial Color-coded Doppler (TCCD)<n>The proposed AAW-YOLO demonstrated strong performance in segmenting both ipsilateral and contralateral CoW vessels.
  arXiv  Detail & Related papers  (2025-08-19T14:41:22Z)
• Predicting Length of Stay in Neurological ICU Patients Using Classical Machine Learning and Neural Network Models: A Benchmark Study on MIMIC-IV [49.1574468325115]
  This study explores multiple ML approaches for predicting LOS in ICU specifically for the patients with neurological diseases based on the MIMIC-IV dataset.<n>The evaluated models include classic ML algorithms (K-Nearest Neighbors, Random Forest, XGBoost and CatBoost) and Neural Networks (LSTM, BERT and Temporal Fusion Transformer)
  arXiv  Detail & Related papers  (2025-05-23T14:06:42Z)
• DeepSeek-R1 Outperforms Gemini 2.0 Pro, OpenAI o1, and o3-mini in Bilingual Complex Ophthalmology Reasoning [4.050281368901217]
  DeepSeek-R1 and three other recently released large ophthalmology reasoning models (LLMs) were evaluated.<n>DeepSeek-R1 achieved the highest accuracy across five topics in both Chinese English and Chinese MCQs.<n> Reasoning ability was evaluated through analyzing reasoning logic and the causes of reasoning error.
  arXiv  Detail & Related papers  (2025-02-25T08:08:53Z)
• CRTRE: Causal Rule Generation with Target Trial Emulation Framework [47.2836994469923]
  We introduce a novel method called causal rule generation with target trial emulation framework (CRTRE) CRTRE applies randomize trial design principles to estimate the causal effect of association rules. We then incorporate such association rules for the downstream applications such as prediction of disease onsets.
  arXiv  Detail & Related papers  (2024-11-10T02:40:06Z)
• Lab-AI: Using Retrieval Augmentation to Enhance Language Models for Personalized Lab Test Interpretation in Clinical Medicine [25.23073509982281]
  Most patient portals use universal normal ranges, ignoring conditional factors like age and gender.<n>This study introduces Lab-AI, an interactive system that offers personalized normal ranges using retrieval-augmented generation (RAG) from credible health sources.
  arXiv  Detail & Related papers  (2024-09-16T20:36:17Z)
• Using Pre-training and Interaction Modeling for ancestry-specific disease prediction in UK Biobank [69.90493129893112]
  Recent genome-wide association studies (GWAS) have uncovered the genetic basis of complex traits, but show an under-representation of non-European descent individuals. Here, we assess whether we can improve disease prediction across diverse ancestries using multiomic data.
  arXiv  Detail & Related papers  (2024-04-26T16:39:50Z)
• IITK at SemEval-2024 Task 2: Exploring the Capabilities of LLMs for Safe Biomedical Natural Language Inference for Clinical Trials [4.679320772294786]
  Large Language models (LLMs) have demonstrated state-of-the-art performance in various natural language processing (NLP) tasks. This research investigates LLMs' robustness, consistency, and faithful reasoning when performing Natural Language Inference (NLI) on breast cancer Clinical Trial Reports (CTRs) We examine the reasoning capabilities of LLMs and their adeptness at logical problem-solving.
  arXiv  Detail & Related papers  (2024-04-06T05:44:53Z)
• Using YOLO v7 to Detect Kidney in Magnetic Resonance Imaging [1.1567496318601842]
  We developed a high-performing model for kidney detection using a semi-supervised approach with a medical image library. Further external validation is required to assess the model's generalizability.
  arXiv  Detail & Related papers  (2024-02-08T16:54:20Z)
• Learning to diagnose cirrhosis from radiological and histological labels with joint self and weakly-supervised pretraining strategies [62.840338941861134]
  We propose to leverage transfer learning from large datasets annotated by radiologists, to predict the histological score available on a small annex dataset. We compare different pretraining methods, namely weakly-supervised and self-supervised ones, to improve the prediction of the cirrhosis. This method outperforms the baseline classification of the METAVIR score, reaching an AUC of 0.84 and a balanced accuracy of 0.75.
  arXiv  Detail & Related papers  (2023-02-16T17:06:23Z)
• A Scalable Workflow to Build Machine Learning Classifiers with Clinician-in-the-Loop to Identify Patients in Specific Diseases [10.658425378457363]
  Clinicians may rely on medical coding systems such as International Classification of Diseases (ICD) to identify patients with diseases from Electronic Health Records (EHRs) Recent studies suggest the ICD codes often cannot characterise patients accurately for specific diseases in real clinical practice. This paper proposes a scalable workflow which leverages both structured data and unstructured textual notes from EHRs with techniques including NLP, AutoML and Clinician-in-the-Loop mechanism.
  arXiv  Detail & Related papers  (2022-05-18T12:24:07Z)
• Automated Quantification of CT Patterns Associated with COVID-19 from Chest CT [48.785596536318884]
  The proposed method takes as input a non-contrasted chest CT and segments the lesions, lungs, and lobes in three dimensions. The method outputs two combined measures of the severity of lung and lobe involvement, quantifying both the extent of COVID-19 abnormalities and presence of high opacities. Evaluation of the algorithm is reported on CTs of 200 participants (100 COVID-19 confirmed patients and 100 healthy controls) from institutions from Canada, Europe and the United States.
  arXiv  Detail & Related papers  (2020-04-02T21:49:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.