Budget-Aware Routing for Long Clinical Text

• URL: http://arxiv.org/abs/2605.00336v1
• Date: Fri, 01 May 2026 01:34:53 GMT
• Title: Budget-Aware Routing for Long Clinical Text
• Abstract summary: Key challenge for large language models is token cost per query and overall deployment cost.<n>We study budgeted context selection, where a subset of document units is chosen under a strict token budget.<n>We propose textbfRCD, a monotone submodular objective that balances relevance, coverage, and diversity.
• Score: 8.474809035213118
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A key challenge for large language models is token cost per query and overall deployment cost. Clinical inputs are long, heterogeneous, and often redundant, while downstream tasks are short and high stakes. We study budgeted context selection, where a subset of document units is chosen under a strict token budget so an off-the-shelf generator can meet fixed cost and latency constraints. We cast this as a knapsack-constrained subset selection problem with two design choices, unitization that defines document segmentation and selection that determines which units are kept. We propose \textbf{RCD}, a monotone submodular objective that balances relevance, coverage, and diversity. We compare sentence, section, window, and cluster-based unitization, and introduce a routing heuristic that adapts to the budget regime. Experiments on MIMIC discharge notes, Cochrane abstracts, and L-Eval show that optimal strategies depend on the evaluation setting. Positional heuristics perform best at low budgets in extractive tasks, while diversity-aware methods such as MMR improve LLM generation. Selector choice matters more than unitization, with cluster-based grouping reducing performance and other schemes behaving similarly. ROUGE saturates for LLM summaries, while BERTScore better reflects quality differences. We release our code at https://github.com/stone-technologies/ACL_budget_paper.

Related papers

• Lightweight Chunk Selection for Mobile Retrieval-Augmented Generation [8.227745781139634]
  We study lightweight RAG chunk selection as an evidence-alignment problem.<n>Our selector combines three complementary feature sources: question hidden states that represent LLM-side query intent, MoE routing-derived expert signals that capture the generator's internal routing structure, and retrieved chunk embeddings that preserve candidate-side evidence geometry.<n>Experiments show that the proposed selector consistently improves rank-1 evidence selection over mobile-applicable baselines by an average of 2.5%.
  arXiv  Detail & Related papers  (2026-08-04T05:26:25Z)
• Efficient Frame Selection for Long Videos at Test Time with Attention-Based MLLM Selectors [69.75895255395702]
  Cross-modal attention at validation-selected extraction layers in MLLMs already provides query-relevant frame evidence without requiring autoregressive generation.<n>A lightweight MLLM selector, even with only 2B parameters, can extract frame-level evidence by converting selected-layer attention into relevance scores through query-conditioned aggregation.<n>Under a 32-frame budget, our selector improves over uniform sampling by up to 6.4 points on Video-MME and outperforms prior training-based selectors under matched frame budgets.
  arXiv  Detail & Related papers  (2026-07-17T07:04:31Z)
• Selective Ensemble Based on Preference-Directed Multi-Objective Bandits [90.75513823660775]
  We formalize a sequential decision problem under partially specified linear preferences.<n>We then propose a preference-directed upper confidence bound (PrefUCB) algorithm.<n> Experiments on large pre-trained model selective ensemble tasks and online asset allocation under institutional mandates validate our method.
  arXiv  Detail & Related papers  (2026-06-20T07:52:46Z)
• Instance-Optimal Estimation with Multiple LLM Judges on a Budget [84.31744861038106]
  We formalize this question as *budgeted heteroskedastic multi-judge estimation*.<n>Given $K$ prompt-response pairs, $J$ judges with known costs, and unknown query-judge variances, the goal is to estimate a bounded score vector while minimizing an $ell_p$-error.<n>We prove that EST-IVWE matches the oracle IVWE rate up to lower-order terms in the budget.
  arXiv  Detail & Related papers  (2026-05-22T08:26:08Z)
• Mixture of Complementary Agents for Robust LLM Ensemble [28.210955621253863]
  Multi-AI collaboration, such as ensembling or debating large language models (LLMs), is a promising paradigm for aggregating information and boosting performance.<n>Existing approaches primarily focus on accuracy (picking the strongest models) or diversity (ensuring variety)<n>We reframe proposer selection as a selection problem akin to feature selection, where the value of an LLM lies in its complementarity with others.
  arXiv  Detail & Related papers  (2026-05-21T21:29:05Z)
• Multinoulli Extension: A Lossless Continuous Relaxation for Partition-Constrained Subset Selection [60.07018090570548]
  We introduce a novel algorithm titled MultinoulliSCG, which is parameterfree and can achieve the same approximation guarantees as the distorted local-search methods.<n>We also present two novel online algorithms, namely, Multinoulli-CG and Multinoulli-GAGA, for the unexplored online subset selection problems over partition constraints.
  arXiv  Detail & Related papers  (2026-03-23T02:30:01Z)
• A Critical Look at Targeted Instruction Selection: Disentangling What Matters (and What Doesn't) [14.070675074621043]
  Instruction fine-tuning involves selecting a subset of instruction training data from a large candidate pool, using a small query set from the target task.<n>Despite growing interest, the literature on targeted instruction selection remains fragmented and opaque.<n>In this work, we aim to bring clarity to this landscape by disentangling and systematically analyzing the two core ingredients: data representation and selection algorithms.
  arXiv  Detail & Related papers  (2026-02-16T12:33:05Z)
• Optimal Budgeted Adaptation of Large Language Models [8.207283283207138]
  Trade-off between labeled data availability and downstream accuracy remains a central challenge in large language models.<n>We propose a principled framework for emphbudget-aware supervised fine-tuning by casting LLM adaptation as a contextual Stackelberg game.
  arXiv  Detail & Related papers  (2026-02-01T01:08:15Z)
• Distilling a Small Utility-Based Passage Selector to Enhance Retrieval-Augmented Generation [110.610512800947]
  Retrieval-augmented generation (RAG) enhances large language models (LLMs) by incorporating retrieved information.<n>In RAG, the emphasis has shifted to utility, which considers the usefulness of passages for generating accurate answers.<n>Our approach focuses on utility-based selection rather than ranking, enabling dynamic passage selection tailored to specific queries without the need for fixed thresholds.<n>Our experiments demonstrate that utility-based selection provides a flexible and cost-effective solution for RAG, significantly reducing computational costs while improving answer quality.
  arXiv  Detail & Related papers  (2025-07-25T09:32:29Z)
• Cequel: Cost-Effective Querying of Large Language Models for Text Clustering [15.179854529085544]
  Text clustering aims to automatically partition a collection of documents into coherent groups based on their linguistic features.<n>Recent advances in large language models (LLMs) have significantly improved this field by providing high-quality contextualized embeddings.<n>We propose Cequel, a cost-effective framework that achieves accurate text clustering under a limited budget of LLM queries.
  arXiv  Detail & Related papers  (2025-04-22T06:57:49Z)
• $Se^2$: Sequential Example Selection for In-Context Learning [83.17038582333716]
  Large language models (LLMs) for in-context learning (ICL) need to be activated by demonstration examples. Prior work has extensively explored the selection of examples for ICL, predominantly following the "select then organize" paradigm. In this paper, we formulate the problem as a $Se$quential $Se$lection problem and introduce $Se2$, a sequential-aware method.
  arXiv  Detail & Related papers  (2024-02-21T15:35:04Z)
• EcoRank: Budget-Constrained Text Re-ranking Using Large Language Models [6.109188517569139]
  We study how to maximize the re-ranking performance given a budget. We propose a suite of budget-constrained methods to perform text re-ranking.
  arXiv  Detail & Related papers  (2024-02-16T18:03:42Z)
• Efficient Prompt Optimization Through the Lens of Best Arm Identification [50.56113809171805]
  This work provides a principled framework, TRIPLE, to efficiently perform prompt selection under an explicit budget constraint. It is built on a novel connection established between prompt optimization and fixed-budget best arm identification (BAI-FB) in multi-armed bandits (MAB)
  arXiv  Detail & Related papers  (2024-02-15T05:31:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.