APRIL: Annotations for Policy evaluation with Reliable Inference from LLMs

• URL: http://arxiv.org/abs/2511.17818v1
• Date: Fri, 21 Nov 2025 22:18:15 GMT
• Title: APRIL: Annotations for Policy evaluation with Reliable Inference from LLMs
• Abstract summary: Off-policy evaluation (OPE) estimates the value of a contextual bandit policy prior to deployment.<n>Previous work has explored using expert-labeled counterfactual annotations to enhance dataset coverage.<n>We propose leveraging large language models (LLMs) to generate counterfactual annotations for OPE in medical domains.
• Score: 1.4349869204948273
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Off-policy evaluation (OPE) estimates the value of a contextual bandit policy prior to deployment. As such, OPE plays a critical role in ensuring safety in high-stakes domains such as healthcare. However, standard OPE approaches are limited by the size and coverage of the behavior dataset. While previous work has explored using expert-labeled counterfactual annotations to enhance dataset coverage, obtaining such annotations is expensive, limiting the scalability of prior approaches. We propose leveraging large language models (LLMs) to generate counterfactual annotations for OPE in medical domains. Our method uses domain knowledge to guide LLMs in predicting how key clinical features evolve under alternate treatments. These predicted features can then be transformed using known reward functions to create counterfactual annotations. We first evaluate the ability of several LLMs to predict clinical features across two patient subsets in MIMIC-IV, finding that state-of-the-art LLMs achieve comparable performance. Building on this capacity to predict clinical features, we generate LLM-based counterfactual annotations and incorporate them into an OPE estimator. Our empirical results analyze the benefits of counterfactual annotations under varying degrees of shift between the behavior and target policies. We find that in most cases, the LLM-based counterfactual annotations significantly improve OPE estimates up to a point. We provide an entropy-based metric to identify when additional annotations cease to be useful. Our results demonstrate that LLM-based counterfactual annotations offer a scalable approach for addressing coverage limitations in healthcare datasets, enabling safer deployment of decision-making policies in clinical settings.

Related papers

• Deployment-Centered Evaluation: Predicting Query-Level Rejection Risk in a Clinical LLM System [33.34100218601891]
  Large language models (LLMs) are increasingly integrated into clinical systems, making it essential to evaluate the real-world utility of these systems.<n> static benchmarks tend to measure correctness rather than user acceptance, aggregate performance across queries, and require densely annotated datasets.<n>We perform a deployment-centered evaluation of an LLM system embedded within electronic health records at an academic medical center, where user feedback is sparse but closely reflects the deployment conditions.
  arXiv  Detail & Related papers  (2026-06-10T21:44:20Z)
• PREBA: Surgical Duration Prediction via PCA-Weighted Retrieval-Augmented LLMs and Bayesian Averaging Aggregation [51.96735866702332]
  PREBA is a retrieval-augmented framework that integrates PCA-weighted retrieval and Bayesian averaging aggregation.<n> PREBA significantly improves performance-for instance, reducing MAE by up to 40% and raising R2 from -0.13 to 0.62 over zero-shot inference.
  arXiv  Detail & Related papers  (2026-02-27T07:19:23Z)
• Evaluating the Pre-Consultation Ability of LLMs using Diagnostic Guidelines [31.28422553778395]
  We introduce EPAG, a benchmark dataset and framework designed for evaluating the pre-consultation ability of LLMs.<n>Small open-source models fine-tuned with a well-curated, task-specific dataset can outperform frontier LLMs in pre-consultation.<n>We find that increased amount of HPI does not necessarily lead to improved diagnostic performance.
  arXiv  Detail & Related papers  (2026-01-07T06:15:21Z)
• Enhancing the Medical Context-Awareness Ability of LLMs via Multifaceted Self-Refinement Learning [49.559151128219725]
  Large language models (LLMs) have shown great promise in the medical domain, achieving strong performance on several benchmarks.<n>However, they continue to underperform in real-world medical scenarios, which often demand stronger context-awareness.<n>We propose Multifaceted Self-Refinement (MuSeR), a data-driven approach that enhances LLMs' context-awareness along three key facets.
  arXiv  Detail & Related papers  (2025-11-13T08:13:23Z)
• Enhancing Rating Prediction with Off-the-Shelf LLMs Using In-Context User Reviews [16.394933051332657]
  Likert-scale rating prediction is a regression task that requires both language and mathematical reasoning to be solved effectively.<n>This study investigates the performance of off-the-shelf LLMs on rating prediction, providing different in-context information.<n>We find that user-written reviews significantly improve the rating prediction performance of LLMs.
  arXiv  Detail & Related papers  (2025-10-01T03:04:20Z)
• Beyond Naïve Prompting: Strategies for Improved Zero-shot Context-aided Forecasting with LLMs [57.82819770709032]
  Large language models (LLMs) can be effective context-aided forecasters via na"ive direct prompting.<n>ReDP improves interpretability by eliciting explicit reasoning traces, allowing us to assess the model's reasoning over the context.<n>CorDP leverages LLMs solely to refine existing forecasts with context, enhancing their applicability in real-world forecasting pipelines.<n> IC-DP proposes embedding historical examples of context-aided forecasting tasks in the prompt, substantially improving accuracy even for the largest models.
  arXiv  Detail & Related papers  (2025-08-13T16:02:55Z)
• Latent Factor Models Meets Instructions: Goal-conditioned Latent Factor Discovery without Task Supervision [50.45597801390757]
  Instruct-LF is a goal-oriented latent factor discovery system.<n>It integrates instruction-following ability with statistical models to handle noisy datasets.
  arXiv  Detail & Related papers  (2025-02-21T02:03:08Z)
• LlaMADRS: Prompting Large Language Models for Interview-Based Depression Assessment [75.44934940580112]
  This study introduces LlaMADRS, a novel framework leveraging open-source Large Language Models (LLMs) to automate depression severity assessment.<n>We employ a zero-shot prompting strategy with carefully designed cues to guide the model in interpreting and scoring transcribed clinical interviews.<n>Our approach, tested on 236 real-world interviews, demonstrates strong correlations with clinician assessments.
  arXiv  Detail & Related papers  (2025-01-07T08:49:04Z)
• ACE-$M^3$: Automatic Capability Evaluator for Multimodal Medical Models [34.81544597731073]
  We introduce ACE-$M3$, an open-sourced textbfAutomatic textbfCapability textbfEvaluator for textbfMultimodal textbfMedical textbfModels.<n>It first utilizes a branch-merge architecture to provide both detailed analysis and a concise final score based on standard medical evaluation criteria.
  arXiv  Detail & Related papers  (2024-12-16T05:15:43Z)
• Improve Temporal Awareness of LLMs for Sequential Recommendation [61.723928508200196]
  Large language models (LLMs) have demonstrated impressive zero-shot abilities in solving a wide range of general-purpose tasks. LLMs fall short in recognizing and utilizing temporal information, rendering poor performance in tasks that require an understanding of sequential data. We propose three prompting strategies to exploit temporal information within historical interactions for LLM-based sequential recommendation.
  arXiv  Detail & Related papers  (2024-05-05T00:21:26Z)
• Dynamic Evaluation of Large Language Models by Meta Probing Agents [44.20074234421295]
  We propose meta probing agents (MPA) to evaluate large language models (LLMs) MPA is the key component of DyVal 2, which naturally extends the previous DyValcitepzhu2023dyval. MPA designs the probing and judging agents to automatically transform an original evaluation problem into a new one following psychometric theory.
  arXiv  Detail & Related papers  (2024-02-21T06:46:34Z)
• Evaluation of General Large Language Models in Contextually Assessing Semantic Concepts Extracted from Adult Critical Care Electronic Health Record Notes [17.648021186810663]
  The purpose of this study was to evaluate the performance of Large Language Models (LLMs) in understanding and processing real-world clinical notes. The GPT family models have demonstrated considerable efficiency, evidenced by their cost-effectiveness and time-saving capabilities.
  arXiv  Detail & Related papers  (2024-01-24T16:52:37Z)
• Retrieving Evidence from EHRs with LLMs: Possibilities and Challenges [18.56314471146199]
  Large volume of notes often associated with patients together with time constraints renders manually identifying relevant evidence practically infeasible. We propose and evaluate a zero-shot strategy for using LLMs as a mechanism to efficiently retrieve and summarize unstructured evidence in patient EHR.
  arXiv  Detail & Related papers  (2023-09-08T18:44:47Z)
• Proximal Reinforcement Learning: Efficient Off-Policy Evaluation in Partially Observed Markov Decision Processes [65.91730154730905]
  In applications of offline reinforcement learning to observational data, such as in healthcare or education, a general concern is that observed actions might be affected by unobserved factors. Here we tackle this by considering off-policy evaluation in a partially observed Markov decision process (POMDP) We extend the framework of proximal causal inference to our POMDP setting, providing a variety of settings where identification is made possible.
  arXiv  Detail & Related papers  (2021-10-28T17:46:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.