Evidence-Gated LLM Priors for Multi-Objective Bayesian Optimization

• URL: http://arxiv.org/abs/2606.01730v1
• Date: Mon, 01 Jun 2026 05:50:16 GMT
• Title: Evidence-Gated LLM Priors for Multi-Objective Bayesian Optimization
• Abstract summary: Large language models (LLMs) are increasingly used as advisors for black-box optimization, yet their suggestions and self-reported confidence are not necessarily calibrated to downstream objective values.<n>We study how to use LLM-generated expert priors in discrete multi-objective Bayesian optimization without blindly trusting them.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) are increasingly used as heuristic advisors for black-box optimization, yet their suggestions and self-reported confidence are not necessarily calibrated to downstream objective values. This issue becomes more pronounced in multi-objective Bayesian optimization, where different objectives may require different expert knowledge and where an LLM expert can be useful for one objective but misleading for another. We study how to use LLM-generated expert priors in discrete multi-objective Bayesian optimization without blindly trusting them. We propose an objective-wise reputation-market mechanism that treats each expert-objective pair as a falsifiable prior source. Expert weights are updated online from observed objective feedback, discounted over time, and gated by market-level trust. We then introduce a decoupled counterfactual gate that can use the LLM prior without confidence, use it with confidence, or abstain from the LLM prior entirely. Across controlled synthetic stress tests and three molecule optimization benchmarks with \qwenflash{}-generated expert priors, we find that dynamic objective-wise calibration improves robustness over fixed LLM priors. However, raw LLM confidence is not reliably beneficial: on ESOL, confidence is positively correlated with prediction error; on FreeSolv, confidence can help; and on Lipophilicity, ignoring confidence remains strongest. Our fixed three-arm counterfactual gate improves over the first counterfactual variant on ESOL and FreeSolv, while an attempted margin portfolio exposes a useful negative result: margin selection should be acquisition-aware rather than based only on one-step prior error.

Related papers

• Instinct vs. Reflection: Unifying Token and Verbalized Confidence in Multimodal Large Models [11.878003218072765]
  Multimodal Large Language Models (MLLMs) have demonstrated exceptional capabilities in various perception and reasoning tasks.<n>We propose a monotone confidence fusion framework to merge dual-channel signals and cross-channel consistency to estimate correctness.<n>Experiments on diverse open-source and closed-source MLLMs show that our method consistently yields more reliable confidence estimates.
  arXiv  Detail & Related papers  (2026-04-19T06:07:39Z)
• Confidence Before Answering: A Paradigm Shift for Efficient LLM Uncertainty Estimation [47.91529693614168]
  Existing methods are predominantly answer-first, producing confidence only after generating an answer.<n>We study a confidence-first paradigm, where the model outputs its confidence before answering, interpreting this score as the model's probability of answering the question correctly.<n>We propose CoCA, a reinforcement learning framework that jointly optimize confidence calibration and answer accuracy via segmented credit assignment.
  arXiv  Detail & Related papers  (2026-03-06T04:03:13Z)
• Reasoning with Confidence: Efficient Verification of LLM Reasoning Steps via Uncertainty Heads [104.9566359759396]
  We propose a lightweight alternative for step-level reasoning verification based on data-driven uncertainty scores.<n>Our findings suggest that the internal states of LLMs encode their uncertainty and can serve as reliable signals for reasoning verification.
  arXiv  Detail & Related papers  (2025-11-09T03:38:29Z)
• MetaFaith: Faithful Natural Language Uncertainty Expression in LLMs [66.14178164421794]
  We introduce MetaFaith, a novel prompt-based calibration approach inspired by human metacognition.<n>We show that MetaFaith robustly improves faithful calibration across diverse models and task domains, enabling up to 61% improvement in faithfulness.
  arXiv  Detail & Related papers  (2025-05-30T17:54:08Z)
• Your Pre-trained LLM is Secretly an Unsupervised Confidence Calibrator [20.81467363927836]
  Post-trained language models (PoLMs) often suffer from over-confidence, assigning high confidence to both correct and incorrect outputs.<n>A major obstacle in calibrating PoLMs is the scarcity of labeled data for individual downstream tasks.<n>We propose Disagreement-Aware Confidence Alignment (DACA) to optimize parameters in post-hoc confidence calibration.
  arXiv  Detail & Related papers  (2025-05-22T13:55:39Z)
• Aligning Large Language Models for Faithful Integrity Against Opposing Argument [71.33552795870544]
  Large Language Models (LLMs) have demonstrated impressive capabilities in complex reasoning tasks.<n>They can be easily misled by unfaithful arguments during conversations, even when their original statements are correct.<n>We propose a novel framework, named Alignment for Faithful Integrity with Confidence Estimation.
  arXiv  Detail & Related papers  (2025-01-02T16:38:21Z)
• Calibrating Large Language Models Using Their Generations Only [44.26441565763495]
  APRICOT is a method to set confidence targets and train an additional model that predicts an LLM's confidence based on its textual input and output alone. It is conceptually simple, does not require access to the target model beyond its output, does not interfere with the language generation, and has a multitude of potential usages. We show how our approach performs competitively in terms of calibration error for white-box and black-box LLMs on closed-book question-answering to detect incorrect LLM answers.
  arXiv  Detail & Related papers  (2024-03-09T17:46:24Z)
• TrustLLM: Trustworthiness in Large Language Models [446.5640421311468]
  This paper introduces TrustLLM, a comprehensive study of trustworthiness in large language models (LLMs) We first propose a set of principles for trustworthy LLMs that span eight different dimensions. Based on these principles, we establish a benchmark across six dimensions including truthfulness, safety, fairness, robustness, privacy, and machine ethics.
  arXiv  Detail & Related papers  (2024-01-10T22:07:21Z)
• Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs [60.61002524947733]
  Previous confidence elicitation methods rely on white-box access to internal model information or model fine-tuning. This leads to a growing need to explore the untapped area of black-box approaches for uncertainty estimation. We define a systematic framework with three components: prompting strategies for eliciting verbalized confidence, sampling methods for generating multiple responses, and aggregation techniques for computing consistency.
  arXiv  Detail & Related papers  (2023-06-22T17:31:44Z)
• Generating with Confidence: Uncertainty Quantification for Black-box Large Language Models [37.63939774027709]
  Large language models (LLMs) specializing in natural language generation (NLG) have recently started exhibiting promising capabilities. We propose and compare several confidence/uncertainty measures, applying them to *selective NLG* where unreliable results could either be ignored or yielded for further assessment. Results reveal that a simple measure for the semantic dispersion can be a reliable predictor of the quality of LLM responses.
  arXiv  Detail & Related papers  (2023-05-30T16:31:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.