Eliciting associations between clinical variables from LLMs via comparison questions across populations

• URL: http://arxiv.org/abs/2605.06335v1
• Date: Thu, 07 May 2026 14:26:32 GMT
• Title: Eliciting associations between clinical variables from LLMs via comparison questions across populations
• Abstract summary: We investigate how it might be possible to recover information on correlation and causal links between patient characteristics.<n>We propose an approach based on structured comparison questions, specifically patient comparison triplet questions.<n>We demonstrate the method in two clinical domains, chronic obstructive pulmonary disease (COPD) and multiple sclerosis (MS)
• Score: 15.876460640820858
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The training data of large language models (LLMs) comprises a wide range of biomedical literature, reflecting data from many different patient populations. We investigate how it might be possible to recover information on correlation and causal links between patient characteristics, as a key building block for medical decision making. To avoid the pitfalls of direct elicitation, we propose an approach based on structured comparison questions, specifically patient comparison triplet questions. This is combined with a statistical model for the LLM representation that provides estimates of correlations without access to activations or model internals. Intuitively, we consider how similarity decisions of LLMs based on a first variable are affected by providing information on a second variable for one of the patients being assessed. We then induce prompt-level environment shifts to obtain correlation estimates for different subpopulations, which enables an invariant causal prediction (ICP) approach to obtain conservative candidate parent links. We demonstrate the method in two clinical domains, chronic obstructive pulmonary disease (COPD) and multiple sclerosis (MS). Across prompted environments, the elicited correlations are smooth, stable, and clinically interpretable, yet vary in a statistically significant way that supports downstream invariance testing, such that ICP provides a small set of candidate invariant parent links. These results show that indirect elicitation via triplet comparisons can recover meaningful association structure from LLMs and offer a cautious route from implicit correlations to causal statements that are congruent with LLM answering patterns.

Related papers

• The Illusion of Intervention: Your LLM-Simulated Experiment is an Observational Study [56.649987197786096]
  Large language models (LLMs) show potential as simulators of human behavior.<n>LLMs can induce unintended shifts in latent user attributes, causing user drift.<n>We formalize the confounding or selection bias that can arise due to user drift.
  arXiv  Detail & Related papers  (2026-05-20T06:09:41Z)
• This Treatment Works, Right? Evaluating LLM Sensitivity to Patient Question Framing in Medical QA [35.372014797343546]
  Large language models (LLMs) are sensitive to prompt phrasings and can be influenced by the way questions are worded.<n>We investigate this through a systematic evaluation in a controlled retrieval-augmented generation (RAG) setting for medical question answering (QA)<n>We construct a dataset of 6,614 query pairs grounded in clinical trial abstracts and evaluate response consistency across eight LLMs.
  arXiv  Detail & Related papers  (2026-04-06T18:03:54Z)
• PREBA: Surgical Duration Prediction via PCA-Weighted Retrieval-Augmented LLMs and Bayesian Averaging Aggregation [51.96735866702332]
  PREBA is a retrieval-augmented framework that integrates PCA-weighted retrieval and Bayesian averaging aggregation.<n> PREBA significantly improves performance-for instance, reducing MAE by up to 40% and raising R2 from -0.13 to 0.62 over zero-shot inference.
  arXiv  Detail & Related papers  (2026-02-27T07:19:23Z)
• On the Credibility of Evaluating LLMs using Survey Questions [0.42061757959666934]
  Recent studies evaluate the value orientation of large language models (LLMs) using adapted social surveys.<n>This paper identifies limitations in this methodology that, depending on the exact setup, can lead to both underestimating and overestimating the similarity of value orientation.<n>Using the World Value Survey in three languages across five countries, we demonstrate that prompting methods (direct vs. chain-of-thought) and decoding strategies (greedy vs. sampling) significantly affect results.
  arXiv  Detail & Related papers  (2026-02-03T21:45:43Z)
• A Federated and Parameter-Efficient Framework for Large Language Model Training in Medicine [59.78991974851707]
  Large language models (LLMs) have demonstrated strong performance on medical benchmarks, including question answering and diagnosis.<n>Most medical LLMs are trained on data from a single institution, which faces limitations in generalizability and safety in heterogeneous systems.<n>We introduce the model-agnostic and parameter-efficient federated learning framework for adapting LLMs to medical applications.
  arXiv  Detail & Related papers  (2026-01-29T18:48:21Z)
• Mind the Gap: Benchmarking LLM Uncertainty, Discrimination, and Calibration in Specialty-Aware Clinical QA [4.501692468580528]
  We evaluate uncertainty estimation methods for clinical question answering (QA) focusing.<n>We present a case study introducing a novel lightweight method based on behavioral features derived from reasoning-oriented models.<n>Our findings reveal that uncertainty reliability is not a monolithic property, but one that depends on clinical specialty and question type.
  arXiv  Detail & Related papers  (2025-06-12T14:48:25Z)
• Uncovering Population PK Covariates from VAE-Generated Latent Spaces [0.24578723416255746]
  We propose a data-driven, model-free framework that integrates Variational Autoencoders (VAEs) deep learning model and LASSO regression.<n>VAE compresses high-dimensional PK signals into a structured latent space, achieving accurate reconstruction with a mean absolute percentage error (MAPE) of 2.26%.
  arXiv  Detail & Related papers  (2025-05-05T09:47:39Z)
• Preference Leakage: A Contamination Problem in LLM-as-a-judge [69.96778498636071]
  Large Language Models (LLMs) as judges and LLM-based data synthesis have emerged as two fundamental LLM-driven data annotation methods.<n>In this work, we expose preference leakage, a contamination problem in LLM-as-a-judge caused by the relatedness between the synthetic data generators and LLM-based evaluators.
  arXiv  Detail & Related papers  (2025-02-03T17:13:03Z)
• A data-driven approach to discover and quantify systemic lupus erythematosus etiological heterogeneity from electronic health records [4.167173990365707]
  Systemic lupus erythematosus (SLE) is a complex disease with many manifestational facets.<n>We propose a data-driven approach to discover probabilistic independent sources from multimodal imperfect EHR data.
  arXiv  Detail & Related papers  (2025-01-13T11:00:31Z)
• LlaMADRS: Prompting Large Language Models for Interview-Based Depression Assessment [75.44934940580112]
  This study introduces LlaMADRS, a novel framework leveraging open-source Large Language Models (LLMs) to automate depression severity assessment.<n>We employ a zero-shot prompting strategy with carefully designed cues to guide the model in interpreting and scoring transcribed clinical interviews.<n>Our approach, tested on 236 real-world interviews, demonstrates strong correlations with clinician assessments.
  arXiv  Detail & Related papers  (2025-01-07T08:49:04Z)
• Uncertainty Estimation of Large Language Models in Medical Question Answering [60.72223137560633]
  Large Language Models (LLMs) show promise for natural language generation in healthcare, but risk hallucinating factually incorrect information. We benchmark popular uncertainty estimation (UE) methods with different model sizes on medical question-answering datasets. Our results show that current approaches generally perform poorly in this domain, highlighting the challenge of UE for medical applications.
  arXiv  Detail & Related papers  (2024-07-11T16:51:33Z)
• Conditional canonical correlation estimation based on covariates with random forests [0.0]
  We propose a new method called Random Forest with Canonical Correlation Analysis (RFCCA) to estimate the conditional canonical correlations between two sets of variables. The proposed method and the global significance test is evaluated through simulation studies that show it provides accurate canonical correlation estimations and well-controlled Type-1 error.
  arXiv  Detail & Related papers  (2020-11-23T17:09:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.