APEX: Automated Prompt Engineering eXpert with Dynamic Data Selection

• URL: http://arxiv.org/abs/2606.11459v1
• Date: Tue, 09 Jun 2026 21:22:06 GMT
• Title: APEX: Automated Prompt Engineering eXpert with Dynamic Data Selection
• Abstract summary: Large Language Models are highly sensitive to prompt formulation, necessitating automatic prompt optimization to unlock their full potential.<n>Current methods treat the development dataset as a static benchmark, wasting significant compute budget on uninformative data.<n>We introduce APEX (Automatic Prompt Engineering eXpert), a novel framework that optimize the data usage alongside the prompt search.
• Score: 60.504476571531
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Models are highly sensitive to prompt formulation, necessitating automatic prompt optimization to unlock their full potential. While evolutionary algorithms have emerged as the dominant paradigm, they suffer from a critical bottleneck: data efficiency. Current methods treat the development dataset as a static benchmark, wasting significant compute budget on uninformative data. In this work, we introduce APEX (Automatic Prompt Engineering eXpert), a novel framework that optimizes the data usage alongside the prompt search. APEX dynamically stratifies the dataset into Easy, Hard, and Mixed tiers based on the optimization lineage. By prioritizing the Mixed tier, which identifies the data where the LLM has mixed performance, we identify two high-leverage subsets: the addressable frontier for generating informative mutations and the rank-sensitive frontier for distinguishing candidate quality. We evaluate APEX across three diverse benchmarks: IFBench, SimpleQA Verified, and FACTS Grounding. Under a fixed budget of 5,000 evaluation calls, due to its data efficiency, APEX outperforms the initial prompt by an average of 11.2% on Gemini 2.5 Flash and 6.8% on Gemma 3 27B, demonstrating that a data-centric approach is key to efficient and effective prompt optimization.

Related papers

• From Insight to Action: A Novel Framework for Interpretability-Guided Data Selection in Large Language Models [73.72877445629383]
  Interpretability-Guided Data Selection (IGDS) is a framework that first identifies causal task features through frequency recall and interventional filtering.<n>We validate IGDS on mathematical reasoning, summarization, and translation tasks within Gemma-2, LLaMA-3.1, and Qwen3 models.
  arXiv  Detail & Related papers  (2026-04-28T03:16:24Z)
• Model Performance-Guided Evaluation Data Selection for Effective Prompt Optimization [12.683042228674694]
  IPOMP is a two-stage approach that selects representative and diverse samples using semantic clustering and boundary analysis.<n>We show that IPOMP improves effectiveness by 1.6% to 5.3% and stability by at least 57% compared with SOTA baselines.
  arXiv  Detail & Related papers  (2025-05-15T22:41:30Z)
• Star-Agents: Automatic Data Optimization with LLM Agents for Instruction Tuning [71.2981957820888]
  We propose a novel Star-Agents framework, which automates the enhancement of data quality across datasets. The framework initially generates diverse instruction data with multiple LLM agents through a bespoke sampling method. The generated data undergo a rigorous evaluation using a dual-model method that assesses both difficulty and quality.
  arXiv  Detail & Related papers  (2024-11-21T02:30:53Z)
• Self-Steering Optimization: Autonomous Preference Optimization for Large Language Models [79.84205827056907]
  We present Self-Steering Optimization ($SSO$), an algorithm that autonomously generates high-quality preference data.<n>$SSO$ employs a specialized optimization objective to build a data generator from the policy model itself, which is used to produce accurate and on-policy data.<n>Our evaluation shows that $SSO$ consistently outperforms baselines in human preference alignment and reward optimization.
  arXiv  Detail & Related papers  (2024-10-22T16:04:03Z)
• 3DS: Medical Domain Adaptation of LLMs via Decomposed Difficulty-based Data Selection [27.922627457625477]
  Large Language Models excel in general tasks but struggle in specialized domains like healthcare.<n>We propose a two-stage model-centric data selection framework, De Difficulty Data Selection (3DS)<n>Our experiments on real-world healthcare datasets demonstrate the superiority of 3DS over exisiting methods in accuracy by over 5.29%.
  arXiv  Detail & Related papers  (2024-10-13T02:29:00Z)
• Cost-Effective Proxy Reward Model Construction with On-Policy and Active Learning [70.22819290458581]
  Reinforcement learning with human feedback (RLHF) is a widely adopted approach in current large language model pipelines. Our approach introduces two key innovations: (1) on-policy query to avoid OOD and imbalance issues in seed data, and (2) active learning to select the most informative data for preference queries.
  arXiv  Detail & Related papers  (2024-07-02T10:09:19Z)
• LESS: Selecting Influential Data for Targeted Instruction Tuning [64.78894228923619]
  We propose LESS, an efficient algorithm to estimate data influences and perform Low-rank gradiEnt Similarity Search for instruction data selection. We show that training on a LESS-selected 5% of the data can often outperform training on the full dataset across diverse downstream tasks. Our method goes beyond surface form cues to identify data that the necessary reasoning skills for the intended downstream application.
  arXiv  Detail & Related papers  (2024-02-06T19:18:04Z)
• Adaptive Prompt Learning-based Few-Shot Sentiment Analysis [6.134988345135517]
  This paper proposes an adaptive prompting(AP) construction strategy using seq2seq-attention structure. The experimental results on FewCLUE datasets demonstrate that the proposed method AP can effectively construct appropriate adaptive prompt.
  arXiv  Detail & Related papers  (2022-05-15T08:34:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.