LatentPrompt: Optimizing Promts in Latent Space

• URL: http://arxiv.org/abs/2508.02452v1
• Date: Mon, 04 Aug 2025 14:17:29 GMT
• Title: LatentPrompt: Optimizing Promts in Latent Space
• Authors: Mateusz Bystroński, Grzegorz Piotrowski, Nitesh V. Chawla, Tomasz Kajdanowicz,
• Abstract summary: We present LatentPrompt, a model-agnostic framework for prompt optimization.<n>Our method embeds seed prompts in a continuous latent space and systematically explores this space to identify prompts that maximize task-specific performance.<n>In a proof-of-concept study on the Financial PhraseBank sentiment classification benchmark, LatentPrompt increased classification accuracy by approximately 3 percent after a single optimization cycle.
• Score: 20.80689930065897
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances have shown that optimizing prompts for Large Language Models (LLMs) can significantly improve task performance, yet many optimization techniques rely on heuristics or manual exploration. We present LatentPrompt, a model-agnostic framework for prompt optimization that leverages latent semantic space to automatically generate, evaluate, and refine candidate prompts without requiring hand-crafted rules. Beginning with a set of seed prompts, our method embeds them in a continuous latent space and systematically explores this space to identify prompts that maximize task-specific performance. In a proof-of-concept study on the Financial PhraseBank sentiment classification benchmark, LatentPrompt increased classification accuracy by approximately 3 percent after a single optimization cycle. The framework is broadly applicable, requiring only black-box access to an LLM and an automatic evaluation metric, making it suitable for diverse domains and tasks.

Related papers

• MOPrompt: Multi-objective Semantic Evolution for Prompt Optimization [0.0699049312989311]
  MOPrompt is a novel framework designed to optimize prompts for both accuracy and context size (measured in tokens) simultaneously.<n>We evaluate MOPrompt on a sentiment analysis task in Portuguese, using Gemma-2B and Sabiazinho-3 as evaluation models.
  arXiv  Detail & Related papers  (2025-08-03T01:50:43Z)
• Grammar-Guided Evolutionary Search for Discrete Prompt Optimisation [63.97051732013936]
  We propose an evolutionary search approach to automated discrete prompt optimisation consisting of two phases.<n>In the first phase, grammar-guided genetic programming is invoked to synthesise prompt-creating programmes.<n>In the second phase, local search is applied to explore the neighbourhoods of best-performing programmes.
  arXiv  Detail & Related papers  (2025-07-14T14:34:15Z)
• TAPO: Task-Referenced Adaptation for Prompt Optimization [18.533289140594146]
  We introduce TAPO, a multitask-aware prompt optimization framework composed of three key modules.<n>First, a task-aware metric selection module is proposed to enhance task-specific prompt generation capabilities.<n>Second, we present a multi-metrics evaluation module to jointly evaluate prompts from multiple perspectives.<n>Third, an evolution-based optimization framework is introduced for automatic prompt refinement, which improves adaptability across various tasks.
  arXiv  Detail & Related papers  (2025-01-12T02:43:59Z)
• A Sequential Optimal Learning Approach to Automated Prompt Engineering in Large Language Models [14.483240353801074]
  This paper proposes an optimal learning framework for automated prompt engineering.<n>It is designed to sequentially identify effective prompt features while efficiently allocating a limited evaluation budget.<n>Our framework provides a solution to deploying automated prompt engineering in a wider range applications.
  arXiv  Detail & Related papers  (2025-01-07T03:51:10Z)
• GReaTer: Gradients over Reasoning Makes Smaller Language Models Strong Prompt Optimizers [52.17222304851524]
  We introduce GReaTer, a novel prompt optimization technique that directly incorporates gradient information over task-specific reasoning.<n>By utilizing task loss gradients, GReaTer enables self-optimization of prompts for open-source, lightweight language models.<n> GReaTer consistently outperforms previous state-of-the-art prompt optimization methods.
  arXiv  Detail & Related papers  (2024-12-12T20:59:43Z)
• Enhancing LLM-Based Text Classification in Political Science: Automatic Prompt Optimization and Dynamic Exemplar Selection for Few-Shot Learning [1.6967824074619953]
  Large language models (LLMs) offer substantial promise for text classification in political science.<n>Our framework enhances LLM performance through automatic prompt optimization, dynamic exemplar selection, and a consensus mechanism.<n>An open-source Python package (PoliPrompt) is available on GitHub.
  arXiv  Detail & Related papers  (2024-09-02T21:05:31Z)
• QPO: Query-dependent Prompt Optimization via Multi-Loop Offline Reinforcement Learning [58.767866109043055]
  We introduce Query-dependent Prompt Optimization (QPO), which iteratively fine-tune a small pretrained language model to generate optimal prompts tailored to the input queries.<n>We derive insights from offline prompting demonstration data, which already exists in large quantities as a by-product of benchmarking diverse prompts on open-sourced tasks.<n> Experiments on various LLM scales and diverse NLP and math tasks demonstrate the efficacy and cost-efficiency of our method in both zero-shot and few-shot scenarios.
  arXiv  Detail & Related papers  (2024-08-20T03:06:48Z)
• MAPO: Boosting Large Language Model Performance with Model-Adaptive Prompt Optimization [73.7779735046424]
  We show that different prompts should be adapted to different Large Language Models (LLM) to enhance their capabilities across various downstream tasks in NLP. We then propose a model-adaptive prompt (MAPO) method that optimize the original prompts for each specific LLM in downstream tasks.
  arXiv  Detail & Related papers  (2024-07-04T18:39:59Z)
• Intent-based Prompt Calibration: Enhancing prompt optimization with synthetic boundary cases [2.6159111710501506]
  We introduce a new method for automatic prompt engineering, using a calibration process that iteratively refines the prompt to the user intent. We demonstrate the effectiveness of our method with respect to strong proprietary models on real-world tasks such as moderation and generation.
  arXiv  Detail & Related papers  (2024-02-05T15:28:43Z)
• Query-Dependent Prompt Evaluation and Optimization with Offline Inverse RL [62.824464372594576]
  We aim to enhance arithmetic reasoning ability of Large Language Models (LLMs) through zero-shot prompt optimization. We identify a previously overlooked objective of query dependency in such optimization. We introduce Prompt-OIRL, which harnesses offline inverse reinforcement learning to draw insights from offline prompting demonstration data.
  arXiv  Detail & Related papers  (2023-09-13T01:12:52Z)
• RLPrompt: Optimizing Discrete Text Prompts With Reinforcement Learning [84.75064077323098]
  This paper proposes RLPrompt, an efficient discrete prompt optimization approach with reinforcement learning (RL) RLPrompt is flexibly applicable to different types of LMs, such as masked gibberish (e.g., grammaBERT) and left-to-right models (e.g., GPTs) Experiments on few-shot classification and unsupervised text style transfer show superior performance over a wide range of existing finetuning or prompting methods.
  arXiv  Detail & Related papers  (2022-05-25T07:50:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.