RASC+: Retrieval-Constrained LLM Adjudication for Clinical Value Set Authoring

• URL: http://arxiv.org/abs/2606.23992v1
• Date: Mon, 22 Jun 2026 22:51:22 GMT
• Title: RASC+: Retrieval-Constrained LLM Adjudication for Clinical Value Set Authoring
• Abstract summary: Clinical code systems are large, version-controlled, and not reliably memorized by language models.<n>We study a stage-wise alternative in which candidate-pool construction is optimized for recall and a constrained LLM adjudicator is optimized for candidate selection.
• Score: 1.066048003460524
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Clinical value sets define the standardized terminology codes used in quality measurement, phenotyping, cohort construction, and clinical decision support. The recently introduced Retrieval-Augmented Set Completion (RASC) benchmark showed that direct zero-shot large language model (LLM) generation is poorly suited to this task: clinical code systems are large, version-controlled, and not reliably memorized by language models. We study a stage-wise alternative in which candidate-pool construction is optimized for recall and a constrained LLM adjudicator is optimized for candidate selection. On the full 3,744-value-set RASC test split, Qwen3-based retrieval with vocabulary-aware expansion and code-display rescue retrieval increases candidate-pool recall from the original RASC retrieval baseline of 0.553 to 0.730; on the held-out-publisher stratum, pool recall is 0.655. The higher-recall pool alone is not sufficient: applying the original SAPBert cross-encoder to this expanded pool gives full-test macro F1 of 0.287 and held-out-publisher macro F1 of 0.233. Replacing the stage-2 selector with blinded GPT-5 adjudication over the same pool increases full-test macro F1 to 0.549 and held-out-publisher macro F1 to 0.533. These results show that retrieval-constrained LLM adjudication can substantially improve value set completion while preserving the safety constraint that all returned codes must come from an auditable candidate pool.

Related papers

• Natural-Language-Guided Generator-Agnostic Shortlisting for Protein Binder Design [16.443448156662644]
  Modern de novo design generate many candidate protein binders, but wet-lab validation capacity remains limited.<n>We study whether LLMs can generate multi-metric ranking policies from precomputed structural-confidence and interface-quality proxy scores.
  arXiv  Detail & Related papers  (2026-08-21T05:32:26Z)
• AssumptionMiner: Extracting, Tracing, and Revising Implicit Assumptions in LLM Code Generation [0.5414847001704249]
  We present AssumptionMiner, a framework that makes implicit assumptions a first-class artifact of large language models (LLMs)<n>In addition to code, AssumptionMiner produces an explicit assumption layer, a structured representation of inferred constraints and design decisions.<n>We evaluate assumption extraction, code localization, and assumption-guided regeneration.
  arXiv  Detail & Related papers  (2026-07-24T20:22:59Z)
• LLM-as-a-Verifier: A General-Purpose Verification Framework [74.40111651545979]
  We introduce LLM-as-a-Verifier, a general-purpose verification framework.<n>It provides fine-grained feedback for agentic tasks without requiring additional training.<n>It achieves state-of-the-art performance on several benchmarks.
  arXiv  Detail & Related papers  (2026-07-06T17:59:35Z)
• Decision-Aware Memory Cards: Counterfactual-Inspired Context Selection and Compression for Tool-Using LLM Agents [3.964533007623828]
  Modern large language model (LLM) agents need decision-relevant evidence at the moment of action.<n>We present the Counterfactual-Inspired Context Layer (CICL), which builds an instance context graph, estimates decision-oriented utility for candidate units, and compresses selected evidence into typed memory cards.<n>CICL provides a practical layer for measuring, ranking, and compressing decision-critical context for tool-using agents.
  arXiv  Detail & Related papers  (2026-06-06T13:02:28Z)
• F-GRPO: Factorized Group-Relative Policy Optimization for Unified Candidate Generation and Ranking [79.49893545611779]
  Large Language Models (LLMs) can generate a subset and order it within a single autoregressive pass.<n>This flexibility introduces a new optimization challenge: the model must search an output space while receiving utility feedback only after the full ranked list is generated.<n>This credit assignment gap makes end-to-end optimization unstable and sample-inefficient.<n>We propose a unified framework that performs both within a single autoregressive rollout and optimize them end-to-end via factorized group-relative policy optimization.
  arXiv  Detail & Related papers  (2026-05-13T04:52:33Z)
• Themis: Training Robust Multilingual Code Reward Models for Flexible Multi-Criteria Scoring [51.497900015187675]
  We develop Themis-CodePreference, the largest open-source collection of code preferences to date, and use it to train Themis-RM, a suite of multilingual code reward models.<n>Our experiments and ablations demonstrate positive scaling trends, strong cross-lingual transfer when training on diverse preferences.
  arXiv  Detail & Related papers  (2026-05-01T16:07:34Z)
• Retrieve, Then Classify: Corpus-Grounded Automation of Clinical Value Set Authoring [1.3108798582758454]
  Clinical value set authoring is a recurring bottleneck in clinical quality measurement and phenotyping.<n>We propose Retrieval-Augmented Set Completion (RASC): retrieve the $K$ most similar existing value sets from a curated corpus to form a candidate pool.<n>We demonstrate the utility of RASC on 11,803 publicly available VSAC value sets, constructing the first large-scale benchmark for this task.
  arXiv  Detail & Related papers  (2026-04-16T04:57:21Z)
• From Stochastic Answers to Verifiable Reasoning: Interpretable Decision-Making with LLM-Generated Code [0.0]
  Large language models (LLMs) are increasingly used for high-stakes decisionmaking.<n>Black-box models obscure their reasoning, while recent LLM-based rule systems rely on per-sample evaluation.<n>We propose reframing LLMs as code generators rather than per-instance evaluators.
  arXiv  Detail & Related papers  (2026-02-28T00:27:29Z)
• Diagnosing LLM-based Rerankers in Cold-Start Recommender Systems: Coverage, Exposure and Practical Mitigations [0.0]
  Large language models (LLMs) and cross-encoder rerankers have gained attention for improving recommender systems.<n>This paper presents a systematic diagnostic study of cross-encoder rerankers in cold-start movie recommendation using the Serendipity-2018 dataset.
  arXiv  Detail & Related papers  (2026-02-09T16:44:42Z)
• ReCALL: Recalibrating Capability Degradation for MLLM-based Composed Image Retrieval [64.14282916266998]
  Composed Image Retrieval aims to retrieve target images based on a hybrid query comprising a reference image and a modification text.<n>We propose ReCALL, a model-agnostic framework that follows a diagnose-generate-refine pipeline.<n>Experiments on CIRR and FashionIQ show that ReCALL consistently recalibrates degraded capabilities and achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-02-02T04:52:54Z)
• Self-reflection in Automated Qualitative Coding: Improving Text Annotation through Secondary LLM Critique [1.5749416770494706]
  Large language models (LLMs) allow for sophisticated qualitative coding of large datasets.<n>We present a simple, generalizable two-stage workflow: an LLM applies a human-designed codebook; a secondary LLM critic performs self-reflection on each positive label.<n>We evaluate this approach on six qualitative codes over 3,000 high-content emails from Apache Software Foundation project evaluation discussions.
  arXiv  Detail & Related papers  (2026-01-14T22:27:13Z)
• Self-Calibrated Listwise Reranking with Large Language Models [137.6557607279876]
  Large language models (LLMs) have been employed in reranking tasks through a sequence-to-sequence approach. This reranking paradigm requires a sliding window strategy to iteratively handle larger candidate sets. We propose a novel self-calibrated listwise reranking method, which aims to leverage LLMs to produce global relevance scores for ranking.
  arXiv  Detail & Related papers  (2024-11-07T10:31:31Z)
• RecDCL: Dual Contrastive Learning for Recommendation [65.6236784430981]
  We propose a dual contrastive learning recommendation framework -- RecDCL. In RecDCL, the FCL objective is designed to eliminate redundant solutions on user-item positive pairs. The BCL objective is utilized to generate contrastive embeddings on output vectors for enhancing the robustness of the representations.
  arXiv  Detail & Related papers  (2024-01-28T11:51:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.