Trustworthy Agents for Electronic Health Records through Confidence Estimation

• URL: http://arxiv.org/abs/2508.19096v1
• Date: Tue, 26 Aug 2025 14:59:04 GMT
• Title: Trustworthy Agents for Electronic Health Records through Confidence Estimation
• Abstract summary: We propose Hallucination Controlled Accuracy at k%, a novel metric quantifying the accuracy-reliability trade-off at varying confidence thresholds.<n>We introduce TrustEHRAgent, a confidence-aware agent incorporating stepwise confidence estimation for clinical question answering.
• Score: 5.313502324941667
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) show promise for extracting information from Electronic Health Records (EHR) and supporting clinical decisions. However, deployment in clinical settings faces challenges due to hallucination risks. We propose Hallucination Controlled Accuracy at k% (HCAcc@k%), a novel metric quantifying the accuracy-reliability trade-off at varying confidence thresholds. We introduce TrustEHRAgent, a confidence-aware agent incorporating stepwise confidence estimation for clinical question answering. Experiments on MIMIC-III and eICU datasets show TrustEHRAgent outperforms baselines under strict reliability constraints, achieving improvements of 44.23%p and 25.34%p at HCAcc@70% while baseline methods fail at these thresholds. These results highlight limitations of traditional accuracy metrics in evaluating healthcare AI agents. Our work contributes to developing trustworthy clinical agents that deliver accurate information or transparently express uncertainty when confidence is low.

Related papers

• When Confidence Fails: Overconfidence in LLMs under Uncertainty and Missing Clinical Information [14.659028787864996]
  Large Language Models (LLMs) have achieved strong performance in medical question answering and clinical reasoning tasks.<n>We conduct a behavioral analysis of LLMs under clinical information uncertainty.<n>We observe significant variation across models in their ability to abstain when the correct answer is unavailable.
  arXiv  Detail & Related papers  (2026-08-10T03:28:46Z)
• CRS-Triage: Confidence- and Reliability-Aware Selective Triage under Incomplete Clinical Evidence [2.5399727721692624]
  Emergency triage requires reliable decisions within a short time period.<n> EHR data, including structured data and clinical text, are often incomplete, unreliable, and inconsistent.<n>We propose confidence- and reliability-aware selective triage (CRS-Triage) to predict patients' acuity levels.
  arXiv  Detail & Related papers  (2026-08-04T16:04:15Z)
• SafeImpute: Reliable Clinical Data Imputation via Conformal Selection [35.40404498771435]
  Clinical care often relies on key laboratory indicators, yet real-world patient visits are sparse and tests are ordered irregularly.<n>We study reliable clinical imputation, aiming to produce accurate imputations while selectively releasing the reliable results.<n>We propose SafeImpute, a reliable imputation framework for irregular and sparse clinical longitudinal records.
  arXiv  Detail & Related papers  (2026-07-06T20:16:21Z)
• Confidence-aware Monocular Depth Estimation for Minimally Invasive Surgery [8.602185221341648]
  endoscopic video sequences are often contaminated by smoke, specular reflections, blur, and occlusions.<n>Current MDE models do not output depth confidence, which could be a valuable tool for improving their clinical reliability.<n>We propose a novel confidence-aware MDE framework featuring three significant contributions.
  arXiv  Detail & Related papers  (2026-03-03T22:59:24Z)
• Guideline-Grounded Evidence Accumulation for High-Stakes Agent Verification [60.18369393468405]
  Existing verifiers usually underperform owing to a lack of domain knowledge and limited calibration.<n>GLEAN compiles expert-curated protocols into trajectory-informed, well-calibrated correctness signals.<n>We empirically validate GLEAN with agentic clinical diagnosis across three diseases from the MIMIC-IV dataset.
  arXiv  Detail & Related papers  (2026-03-03T09:36:43Z)
• Towards Reliable Medical LLMs: Benchmarking and Enhancing Confidence Estimation of Large Language Models in Medical Consultation [97.36081721024728]
  We propose the first benchmark for assessing confidence in multi-turn interaction during realistic medical consultations.<n>Our benchmark unifies three types of medical data for open-ended diagnostic generation.<n>We present MedConf, an evidence-grounded linguistic self-assessment framework.
  arXiv  Detail & Related papers  (2026-01-22T04:51:39Z)
• MedBayes-Lite: Bayesian Uncertainty Quantification for Safe Clinical Decision Support [5.22077647816926]
  We propose MedBayes-Lite, a lightweight Bayesian enhancement for transformer-based clinical language models.<n> MedBayes-Lite embeds uncertainty quantification directly into existing transformer pipelines without any retraining or architectural rewiring.<n>It consistently improves calibration and trustworthiness, reducing overconfidence by 32 to 48 percent.
  arXiv  Detail & Related papers  (2025-11-20T18:33:12Z)
• Mind the Generation Process: Fine-Grained Confidence Estimation During LLM Generation [63.49409574310576]
  Large language models (LLMs) exhibit overconfidence, assigning high confidence scores to incorrect predictions.<n>We introduce FineCE, a novel confidence estimation method that delivers accurate, fine-grained confidence scores during text generation.<n>Our code and all baselines used in the paper are available on GitHub.
  arXiv  Detail & Related papers  (2025-08-16T13:29:35Z)
• Beyond Benchmarks: Dynamic, Automatic And Systematic Red-Teaming Agents For Trustworthy Medical Language Models [87.66870367661342]
  Large language models (LLMs) are used in AI applications in healthcare.<n>Red-teaming framework that continuously stress-test LLMs can reveal significant weaknesses in four safety-critical domains.<n>A suite of adversarial agents is applied to autonomously mutate test cases, identify/evolve unsafe-triggering strategies, and evaluate responses.<n>Our framework delivers an evolvable, scalable, and reliable safeguard for the next generation of medical AI.
  arXiv  Detail & Related papers  (2025-07-30T08:44:22Z)
• Fact-Level Confidence Calibration and Self-Correction [64.40105513819272]
  We propose a Fact-Level framework that calibrates confidence to relevance-weighted correctness at the fact level. We also develop Confidence-Guided Fact-level Self-Correction ($textbfConFix$), which uses high-confidence facts within a response as additional knowledge to improve low-confidence ones.
  arXiv  Detail & Related papers  (2024-11-20T14:15:18Z)
• Towards Reliable Medical Image Segmentation by utilizing Evidential Calibrated Uncertainty [52.03490691733464]
  We introduce DEviS, an easily implementable foundational model that seamlessly integrates into various medical image segmentation networks. By leveraging subjective logic theory, we explicitly model probability and uncertainty for the problem of medical image segmentation. DeviS incorporates an uncertainty-aware filtering module, which utilizes the metric of uncertainty-calibrated error to filter reliable data.
  arXiv  Detail & Related papers  (2023-01-01T05:02:46Z)
• Confidence-Calibrated Face and Kinship Verification [8.570969129199467]
  We introduce an effective confidence measure that allows verification models to convert a similarity score into a confidence score for any given face pair. We also propose a confidence-calibrated approach, termed Angular Scaling (ASC), which is easy to implement and can be readily applied to existing verification models. To the best of our knowledge, our work presents the first comprehensive confidence-calibrated solution for modern face and kinship verification tasks.
  arXiv  Detail & Related papers  (2022-10-25T10:43:46Z)
• UNITE: Uncertainty-based Health Risk Prediction Leveraging Multi-sourced Data [81.00385374948125]
  We present UNcertaInTy-based hEalth risk prediction (UNITE) model. UNITE provides accurate disease risk prediction and uncertainty estimation leveraging multi-sourced health data. We evaluate UNITE on real-world disease risk prediction tasks: nonalcoholic fatty liver disease (NASH) and Alzheimer's disease (AD) UNITE achieves up to 0.841 in F1 score for AD detection, up to 0.609 in PR-AUC for NASH detection, and outperforms various state-of-the-art baselines by up to $19%$ over the best baseline.
  arXiv  Detail & Related papers  (2020-10-22T02:28:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.