Embedding by Elicitation: Dynamic Representations for Bayesian Optimization of System Prompts

• URL: http://arxiv.org/abs/2605.19093v1
• Date: Mon, 18 May 2026 20:28:17 GMT
• Title: Embedding by Elicitation: Dynamic Representations for Bayesian Optimization of System Prompts
• Abstract summary: System prompts are a central control mechanism in modern AI systems, shaping behavior across conversations, tasks, and user populations.<n>We study this aggregate feedback setting as sample-constrained black-box optimization over discrete, variable-length text.<n>We introduce ReElicit, a Bayesian optimization framework based on emphembedding by elicitation.
• Score: 22.060204303535187
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: System prompts are a central control mechanism in modern AI systems, shaping behavior across conversations, tasks, and user populations. Yet they are difficult to tune when feedback is available only as aggregate metrics rather than per-example labels, failures, or critiques. We study this aggregate feedback setting as sample-constrained black-box optimization over discrete, variable-length text. We introduce ReElicit, a Bayesian optimization framework based on \emph{embedding by elicitation}. Given a task description, previously evaluated prompts, and scalar scores, an LLM elicits a compact, interpretable feature space and maps prompts into it. Leveraging a probabilistic Gaussian process surrogate, an acquisition function then selects target feature vectors, which the LLM realizes and refines into deployable system prompts. Re-eliciting the feature space as new evaluations arrive lets the representation adapt to the observed prompt-score history. We evaluate the setting using offline benchmark accuracy as a controlled aggregate proxy: the optimizer observes one scalar score per prompt and no per-example labels, errors, or critiques. Across ten system prompt optimization tasks with a 30 total evaluation budget, ReElicit achieves the strongest aggregate performance profile among representative aggregate-only prompt-optimization baselines. These results suggest that LLMs can serve as adaptive semantic representation builders, not only prompt generators, for Bayesian optimization over natural-language artifacts.

Related papers

• BayesPO: Bayesian Prompt Optimization via Parallel-Tempered Gradient-Guided Discrete MCMC [10.650418686153921]
  BayesPO is instantiated with Markov chain Monte Carlo and integrates parallel tempering for global exploration of rugged energy landscapes.<n> Experiments with Qwen2.5 models show that the sampler discovers semantically meaningful prompts on diagnostic tasks.<n>Post-optimizing APE prompts on 24 instruction-induction subtasks improves average accuracy from 60.04% to 63.23%.
  arXiv  Detail & Related papers  (2026-07-17T14:39:05Z)
• GFlowPO: Generative Flow Network as a Language Model Prompt Optimizer [51.31263673158136]
  GFlowPO casts prompt search as a posterior inference problem over latent prompts regularized by a meta-prompted reference-LM prior.<n>GFlowPO consistently outperforms recent discrete prompt optimization baselines.
  arXiv  Detail & Related papers  (2026-02-03T10:30:03Z)
• Automatic Prompt Optimization for Dataset-Level Feature Discovery [38.37728428959515]
  We formulate feature discovery as a dataset-level prompt optimization problem.<n>We propose a multi-agent prompt optimization framework in which language-model agents jointly propose feature definitions, extract feature values, and evaluate feature quality.<n>This formulation departs from prior prompt optimization methods that rely on per-sample supervision and provides a principled mechanism for automatic feature discovery from unstructured text.
  arXiv  Detail & Related papers  (2026-01-20T12:51:03Z)
• Auto-Prompting with Retrieval Guidance for Frame Detection in Logistics [0.0]
  Prompt engineering plays a critical role in adapting large language models (LLMs) to complex reasoning and labeling tasks without the need for extensive fine-tuning.<n>We propose a novel prompt optimization pipeline for frame detection in logistics texts, combining retrieval-augmented generation (RAG), few-shot prompting, chain-of-thought (CoT) reasoning, and automatic CoT synthesis (Auto-CoT) to generate highly effective task-specific prompts.<n>Our framework is evaluated on a real-world logistics text annotation task, where reasoning accuracy and labeling efficiency are critical.
  arXiv  Detail & Related papers  (2025-12-22T10:29:51Z)
• LLM Based Bayesian Optimization for Prompt Search [6.764478031814792]
  We propose an algorithm for prompt engineering to enhance text classification with Large Language Models.<n>The proposed BO-LLM algorithm is evaluated on two datasets, and its advantages are discussed in this paper.
  arXiv  Detail & Related papers  (2025-10-05T22:32:50Z)
• Self-Calibrated Listwise Reranking with Large Language Models [137.6557607279876]
  Large language models (LLMs) have been employed in reranking tasks through a sequence-to-sequence approach. This reranking paradigm requires a sliding window strategy to iteratively handle larger candidate sets. We propose a novel self-calibrated listwise reranking method, which aims to leverage LLMs to produce global relevance scores for ranking.
  arXiv  Detail & Related papers  (2024-11-07T10:31:31Z)
• In-context Demonstration Matters: On Prompt Optimization for Pseudo-Supervision Refinement [71.60563181678323]
  Large language models (LLMs) have achieved great success across diverse tasks, and fine-tuning is sometimes needed to further enhance generation quality.<n>To handle these challenges, a direct solution is to generate high-confidence'' data from unsupervised downstream tasks.<n>We propose a novel approach, pseudo-supervised demonstrations aligned prompt optimization (PAPO) algorithm, which jointly refines both the prompt and the overall pseudo-supervision.
  arXiv  Detail & Related papers  (2024-10-04T03:39:28Z)
• Prompt Optimization with EASE? Efficient Ordering-aware Automated Selection of Exemplars [66.823588073584]
  Large language models (LLMs) have shown impressive capabilities in real-world applications. The quality of these exemplars in the prompt greatly impacts performance. Existing methods fail to adequately account for the impact of exemplar ordering on the performance.
  arXiv  Detail & Related papers  (2024-05-25T08:23:05Z)
• Relative Preference Optimization: Enhancing LLM Alignment through Contrasting Responses across Identical and Diverse Prompts [95.09994361995389]
  Relative Preference Optimization (RPO) is designed to discern between more and less preferred responses derived from both identical and related prompts. RPO has demonstrated a superior ability to align large language models with user preferences and to improve their adaptability during the training process.
  arXiv  Detail & Related papers  (2024-02-12T22:47:57Z)
• Robust Prompt Optimization for Large Language Models Against Distribution Shifts [80.6757997074956]
  Large Language Model (LLM) has demonstrated significant ability in various Natural Language Processing tasks. We propose a new problem of robust prompt optimization for LLMs against distribution shifts. This problem requires the prompt optimized over the labeled source group can simultaneously generalize to an unlabeled target group.
  arXiv  Detail & Related papers  (2023-05-23T11:30:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.