When Do We Need LLMs? A Diagnostic for Language-Driven Bandits

• URL: http://arxiv.org/abs/2604.05859v1
• Date: Tue, 07 Apr 2026 13:20:06 GMT
• Title: When Do We Need LLMs? A Diagnostic for Language-Driven Bandits
• Abstract summary: We study Contextual Multi-Armed Bandits (CMABs) for non-episodic sequential decision making problems in finance.<n>Our results provide a principled deployment framework for cost-effective, uncertainty-aware decision systems.
• Score: 44.62361611699998
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We study Contextual Multi-Armed Bandits (CMABs) for non-episodic sequential decision making problems where the context includes both textual and numerical information (e.g., recommendation systems, dynamic portfolio adjustments, offer selection; all frequent problems in finance). While Large Language Models (LLMs) are increasingly applied to these settings, utilizing LLMs for reasoning at every decision step is computationally expensive and uncertainty estimates are difficult to obtain. To address this, we introduce LLMP-UCB, a bandit algorithm that derives uncertainty estimates from LLMs via repeated inference. However, our experiments demonstrate that lightweight numerical bandits operating on text embeddings (dense or Matryoshka) match or exceed the accuracy of LLM-based solutions at a fraction of their cost. We further show that embedding dimensionality is a practical lever on the exploration-exploitation balance, enabling cost--performance tradeoffs without prompt complexity. Finally, to guide practitioners, we propose a geometric diagnostic based on the arms' embedding to decide when to use LLM-driven reasoning versus a lightweight numerical bandit. Our results provide a principled deployment framework for cost-effective, uncertainty-aware decision systems with broad applicability across AI use cases in financial services.

Related papers

• An Empirical Study of LLM Reasoning Ability Under Strict Output Length Constraint [20.08804908112807]
  It is unclear whether and how the reasoning ability of different Large Language Models remain effective under strict constraints.<n>We test 30 LLMs on common reasoning datasets under a wide range of output length budgets.<n>The results have demonstrated several interesting findings regarding the budget-aware LLM reasoning ability.
  arXiv  Detail & Related papers  (2025-04-19T16:32:28Z)
• Supervised Optimism Correction: Be Confident When LLMs Are Sure [91.7459076316849]
  We establish a novel theoretical connection between supervised fine-tuning and offline reinforcement learning.<n>We show that the widely used beam search method suffers from unacceptable over-optimism.<n>We propose Supervised Optimism Correction, which introduces a simple yet effective auxiliary loss for token-level $Q$-value estimations.
  arXiv  Detail & Related papers  (2025-04-10T07:50:03Z)
• Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making [85.24399869971236]
  We aim to evaluate Large Language Models (LLMs) for embodied decision making.<n>Existing evaluations tend to rely solely on a final success rate.<n>We propose a generalized interface (Embodied Agent Interface) that supports the formalization of various types of tasks.
  arXiv  Detail & Related papers  (2024-10-09T17:59:00Z)
• EVOLvE: Evaluating and Optimizing LLMs For In-Context Exploration [76.66831821738927]
  Large language models (LLMs) remain under-studied in scenarios requiring optimal decision-making under uncertainty.<n>We measure LLMs' (in)ability to make optimal decisions in bandits, a state-less reinforcement learning setting relevant to many applications.<n>Motivated by the existence of optimal exploration algorithms, we propose efficient ways to integrate this algorithmic knowledge into LLMs.
  arXiv  Detail & Related papers  (2024-10-08T17:54:03Z)
• CLAMBER: A Benchmark of Identifying and Clarifying Ambiguous Information Needs in Large Language Models [60.59638232596912]
  We introduce CLAMBER, a benchmark for evaluating large language models (LLMs) Building upon the taxonomy, we construct 12K high-quality data to assess the strengths, weaknesses, and potential risks of various off-the-shelf LLMs. Our findings indicate the limited practical utility of current LLMs in identifying and clarifying ambiguous user queries.
  arXiv  Detail & Related papers  (2024-05-20T14:34:01Z)
• Efficient Contextual LLM Cascades through Budget-Constrained Policy Learning [31.972053219549757]
  TREACLE is a reinforcement learning policy that jointly selects the model and prompting scheme while respecting the user's monetary cost and latency constraints. Our evaluations show that TREACLE enables cost savings of up to 85% compared to baselines, while maintaining high accuracy.
  arXiv  Detail & Related papers  (2024-04-17T05:56:49Z)
• FAC$^2$E: Better Understanding Large Language Model Capabilities by Dissociating Language and Cognition [56.76951887823882]
  Large language models (LLMs) are primarily evaluated by overall performance on various text understanding and generation tasks. We present FAC$2$E, a framework for Fine-grAined and Cognition-grounded LLMs' Capability Evaluation.
  arXiv  Detail & Related papers  (2024-02-29T21:05:37Z)
• On Leveraging Large Language Models for Enhancing Entity Resolution: A Cost-efficient Approach [7.996010840316654]
  We propose an uncertainty reduction framework using Large Language Models (LLMs) to improve entity resolution results. LLMs capitalize on their advanced linguistic capabilities and a pay-as-you-go'' model that provides significant advantages to those without extensive data science expertise. We show that our method is efficient and effective, offering promising applications in real-world tasks.
  arXiv  Detail & Related papers  (2024-01-07T09:06:58Z)
• Zero-Shot Question Answering over Financial Documents using Large Language Models [0.18749305679160366]
  We introduce a large language model (LLM) based approach to answer complex questions requiring multi-hop numerical reasoning over financial reports. We use novel zero-shot prompts that guide the LLM to encode the required reasoning into a Python program or a domain specific language.
  arXiv  Detail & Related papers  (2023-11-19T16:23:34Z)
• Large Language Models in Finance: A Survey [12.243277149505364]
  Large language models (LLMs) have opened new possibilities for artificial intelligence applications in finance. Recent advances in large language models (LLMs) have opened new possibilities for artificial intelligence applications in finance.
  arXiv  Detail & Related papers  (2023-09-28T06:04:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.