Teach Better or Show Smarter? On Instructions and Exemplars in Automatic Prompt Optimization

• URL: http://arxiv.org/abs/2406.15708v2
• Date: Wed, 06 Nov 2024 22:07:17 GMT
• Title: Teach Better or Show Smarter? On Instructions and Exemplars in Automatic Prompt Optimization
• Authors: Xingchen Wan, Ruoxi Sun, Hootan Nakhost, Sercan O. Arik,
• Abstract summary: This paper comprehensively compares the performance of representative IO and EO techniques on a diverse set of challenging tasks. We find that intelligently reusing model-generated input-output pairs consistently improves performance on top of IO methods. We also observe a synergy between EO and IO, with optimal combinations surpassing the individual contributions.
• Score: 15.967049403803749
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models have demonstrated remarkable capabilities, but their performance is heavily reliant on effective prompt engineering. Automatic prompt optimization (APO) methods are designed to automate this and can be broadly categorized into those targeting instructions (instruction optimization, IO) vs. those targeting exemplars (exemplar optimization, EO). Despite their shared objective, these have evolved rather independently, with IO receiving more research attention recently. This paper seeks to bridge this gap by comprehensively comparing the performance of representative IO and EO techniques both isolation and combination on a diverse set of challenging tasks. Our findings reveal that intelligently reusing model-generated input-output pairs obtained from evaluating prompts on the validation set as exemplars, consistently improves performance on top of IO methods but is currently under-investigated. We also find that despite the recent focus on IO, how we select exemplars can outweigh how we optimize instructions, with EO strategies as simple as random search outperforming state-of-the-art IO methods with seed instructions without any optimization. Moreover, we observe a synergy between EO and IO, with optimal combinations surpassing the individual contributions. We conclude that studying exemplar optimization both as a standalone method and its optimal combination with instruction optimization remain a crucial aspect of APO and deserve greater consideration in future research, even in the era of highly capable instruction-following models.

Related papers

• ELPO: Ensemble Learning Based Prompt Optimization for Large Language Models [39.71820341519503]
  We propose a novel framework called Ensemble Learning based Prompt Optimization (ELPO) to achieve more accurate and robust results.<n>Motivated by the idea of ensemble learning, ELPO conducts voting mechanism and introduces shared generation strategies.<n>ELPO creatively presents more efficient algorithms for the prompt generation and search process.
  arXiv  Detail & Related papers  (2025-11-20T07:27:26Z)
• Better by Comparison: Retrieval-Augmented Contrastive Reasoning for Automatic Prompt Optimization [6.3914079241545885]
  We present Contrastive Reasoning Prompt Optimization (CRPO), a novel framework that formulates prompt optimization as a retrieval-augmented reasoning process.<n>Our approach retrieves top k reference prompt-response pairs from the HelpSteer2 dataset.<n>By explicitly contrasting high and low quality exemplars, CRPO enables the model to deduce why certain prompts succeed while others fail.
  arXiv  Detail & Related papers  (2025-09-02T08:45:29Z)
• ORPP: Self-Optimizing Role-playing Prompts to Enhance Language Model Capabilities [64.24517317344959]
  High-quality prompts are crucial for eliciting outstanding performance from large language models on complex tasks.<n>We propose ORPP, a framework that enhances model performance by optimizing and generating role-playing prompts.<n>We show that ORPP not only matches but in most cases surpasses existing mainstream prompt optimization methods in terms of performance.
  arXiv  Detail & Related papers  (2025-06-03T05:51:35Z)
• In-context Ranking Preference Optimization [65.5489745857577]
  We propose an In-context Ranking Preference Optimization (IRPO) framework to optimize large language models (LLMs) based on ranking lists constructed during inference.<n>We show IRPO outperforms standard DPO approaches in ranking performance, highlighting its effectiveness in aligning LLMs with direct in-context ranking preferences.
  arXiv  Detail & Related papers  (2025-04-21T23:06:12Z)
• RAISE: Reinforenced Adaptive Instruction Selection For Large Language Models [48.63476198469349]
  We propose a task-objective-driven instruction selection framework RAISE. RAISE incorporates the entire instruction fine-tuning process into optimization. It selects instruction at each step based on the expected impact of instruction on model performance improvement.
  arXiv  Detail & Related papers  (2025-04-09T21:17:52Z)
• Review, Refine, Repeat: Understanding Iterative Decoding of AI Agents with Dynamic Evaluation and Selection [71.92083784393418]
  Inference-time methods such as Best-of-N (BON) sampling offer a simple yet effective alternative to improve performance. We propose Iterative Agent Decoding (IAD) which combines iterative refinement with dynamic candidate evaluation and selection guided by a verifier.
  arXiv  Detail & Related papers  (2025-04-02T17:40:47Z)
• ToolACE-R: Tool Learning with Adaptive Self-Refinement [84.69651852838794]
  Tool learning allows Large Language Models to leverage external tools for solving complex user tasks. We propose ToolACE-R, a novel method that introduces adaptive self-refinement for tool invocations. Our results demonstrate the effectiveness of the proposed method, which is compatible with base models of various sizes.
  arXiv  Detail & Related papers  (2025-04-02T06:38:56Z)
• Evolutionary Pre-Prompt Optimization for Mathematical Reasoning [45.461506988071534]
  This paper explores the optimization of example selection for designing effective chain-of-thought pre-prompts. It shows that the choice of the algorithm, typically in favor of comparison-based methods such as evolutionary computation, significantly enhances efficacy and feasibility.
  arXiv  Detail & Related papers  (2024-12-05T16:12:06Z)
• Dynamic Rewarding with Prompt Optimization Enables Tuning-free Self-Alignment of Language Models [54.381650481255235]
  We introduce a new tuning-free approach for self-alignment, Dynamic Rewarding with Prompt Optimization (O) Our approach leverages a search-based optimization framework that allows LLMs to iteratively self-improve and craft the optimal alignment instructions. Empirical evaluations on eight recent LLMs, both open and closed-sourced, demonstrate that DRPO significantly enhances alignment performance.
  arXiv  Detail & Related papers  (2024-11-13T16:15:38Z)
• IOPO: Empowering LLMs with Complex Instruction Following via Input-Output Preference Optimization [74.34707794886751]
  This paper introduces TRACE, a benchmark for improving and evaluating the complex instructionfollowing ability. We also propose IOPO, which takes both input and output preference pairs into consideration. Experiments on both in-domain and out-of-domain datasets confirm the effectiveness of IOPO.
  arXiv  Detail & Related papers  (2024-11-09T15:12:43Z)
• Self-supervised Preference Optimization: Enhance Your Language Model with Preference Degree Awareness [27.43137305486112]
  We propose a novel Self-supervised Preference Optimization (SPO) framework, which constructs a self-supervised preference degree loss combined with the alignment loss. The results demonstrate that SPO can be seamlessly integrated with existing preference optimization methods to achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-09-26T12:37:26Z)
• AIPO: Improving Training Objective for Iterative Preference Optimization [34.24211649396053]
  We study iterative preference optimization with synthetic data. We propose our training objective for iterative preference optimization, namely Agreement-aware Iterative Preference Optimization (AIPO)
  arXiv  Detail & Related papers  (2024-09-13T14:03:49Z)
• Large Language Models Prompting With Episodic Memory [53.8690170372303]
  We propose PrOmpting with Episodic Memory (POEM), a novel prompt optimization technique that is simple, efficient, and demonstrates strong generalization capabilities. In the testing phase, we optimize the sequence of examples for each test query by selecting the sequence that yields the highest total rewards from the top-k most similar training examples in the episodic memory. Our results show that POEM outperforms recent techniques like TEMPERA and RLPrompt by over 5.3% in various text classification tasks.
  arXiv  Detail & Related papers  (2024-08-14T11:19:28Z)
• Prompt Optimization with EASE? Efficient Ordering-aware Automated Selection of Exemplars [66.823588073584]
  Large language models (LLMs) have shown impressive capabilities in real-world applications. The quality of these exemplars in the prompt greatly impacts performance. Existing methods fail to adequately account for the impact of exemplar ordering on the performance.
  arXiv  Detail & Related papers  (2024-05-25T08:23:05Z)
• FIPO: Free-form Instruction-oriented Prompt Optimization with Preference Dataset and Modular Fine-tuning Schema [36.65009632307124]
  We propose Free-from Instruction-oriented Prompt Optimization (FIPO) to improve task performance of large language models (LLMs) FIPO uses a modular APO template that dynamically integrate the naive task instruction, optional instruction responses, and optional ground truth to produce finely optimized prompts. We validate FIPO framework across five public benchmarks and six testing models.
  arXiv  Detail & Related papers  (2024-02-19T03:56:44Z)
• PhaseEvo: Towards Unified In-Context Prompt Optimization for Large Language Models [9.362082187605356]
  We present PhaseEvo, an efficient automatic prompt optimization framework that combines the generative capability of LLMs with the global search proficiency of evolution algorithms. PhaseEvo significantly outperforms the state-of-the-art baseline methods by a large margin whilst maintaining good efficiency.
  arXiv  Detail & Related papers  (2024-02-17T17:47:10Z)
• Query-Dependent Prompt Evaluation and Optimization with Offline Inverse RL [62.824464372594576]
  We aim to enhance arithmetic reasoning ability of Large Language Models (LLMs) through zero-shot prompt optimization. We identify a previously overlooked objective of query dependency in such optimization. We introduce Prompt-OIRL, which harnesses offline inverse reinforcement learning to draw insights from offline prompting demonstration data.
  arXiv  Detail & Related papers  (2023-09-13T01:12:52Z)
• Robust Prompt Optimization for Large Language Models Against Distribution Shifts [80.6757997074956]
  Large Language Model (LLM) has demonstrated significant ability in various Natural Language Processing tasks. We propose a new problem of robust prompt optimization for LLMs against distribution shifts. This problem requires the prompt optimized over the labeled source group can simultaneously generalize to an unlabeled target group.
  arXiv  Detail & Related papers  (2023-05-23T11:30:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.