Adaptive Prompt Structure Factorization: A Framework for Self-Discovering and Optimizing Compositional Prompt Programs

• URL: http://arxiv.org/abs/2604.06699v1
• Date: Wed, 08 Apr 2026 05:25:40 GMT
• Title: Adaptive Prompt Structure Factorization: A Framework for Self-Discovering and Optimizing Compositional Prompt Programs
• Abstract summary: We propose Adaptive Prompt Structure Factorization (aPSF), an API-only framework (prompt-in/text-out; no access to model internals)<n>aPSF uses an Architect model to discover task-specific prompt structures as semantic factors.<n>aPSF outperforms strong baselines including principle-awares, improving accuracy by up to +2.16 percentage points on average.
• Score: 18.579725648410815
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automated prompt optimization is crucial for eliciting reliable reasoning from large language models (LLMs), yet most API-only prompt optimizers iteratively edit monolithic prompts, coupling components and obscuring credit assignment, limiting controllability, and wasting tokens. We propose Adaptive Prompt Structure Factorization (aPSF), an API-only framework (prompt-in/text-out; no access to model internals) that uses an Architect model to discover task-specific prompt structures as semantic factors. aPSF then performs interventional, single-factor updates: interventional factor-level scoring estimates each factor's marginal contribution via validation-performance changes, and error-guided factor selection routes updates to the current dominant failure source for more sample-efficient optimization. Across multiple advanced reasoning benchmarks, aPSF outperforms strong baselines including principle-aware optimizers, improving accuracy by up to +2.16 percentage points on average, and reduces optimization cost by 45--87% tokens on MultiArith while reaching peak validation in 1 step.

Related papers

• AdaExplore: Failure-Driven Adaptation and Diversity-Preserving Search for Efficient Kernel Generation [59.964989458924585]
  AdaExplore is an agent framework that enables self-improvement via accumulated execution feedback for performance-critical kernel code generation.<n>In the adaptation stage, the agent synthesizes tasks and converts recurring failures into a reusable memory of validity rules.<n>In the search stage, the agent organizes candidate kernels as a tree and alternates between small local refinements and larger structural regeneration.
  arXiv  Detail & Related papers  (2026-04-17T18:25:03Z)
• CROP: Token-Efficient Reasoning in Large Language Models via Regularized Prompt Optimization [0.0]
  Large Language Models utilizing reasoning techniques improve task performance but incur significant latency and token costs due to verbose generation.<n>We propose Cost-Regularized Optimization of Prompts (CROP), an APO method that introduces regularization on response length.<n>We evaluate our approach on complex reasoning datasets, specifically GSM8K, LogiQA and BIG-Bench Hard.
  arXiv  Detail & Related papers  (2026-04-08T19:31:25Z)
• Optimizing LLM Prompt Engineering with DSPy Based Declarative Learning [0.0]
  Large Language Models (LLMs) have shown strong performance across a wide range of natural language processing tasks.<n>Their effectiveness is highly dependent on prompt design, structure, and embedded reasoning.<n>This paper presents a systematic study of DSPy-based declarative learning for prompt optimization.
  arXiv  Detail & Related papers  (2026-04-06T17:17:57Z)
• CODMAS: A Dialectic Multi-Agent Collaborative Framework for Structured RTL Optimization [5.592208774984063]
  CODMAS is a framework that combines structured dialectic reasoning with domain-aware code generation and deterministic evaluation.<n>We introduce RTLOPT, a benchmark of 120 Verilog triples (unoptimized, optimized, testbench) for pipelining and clock-gating transformations.
  arXiv  Detail & Related papers  (2026-03-17T23:10:07Z)
• HIPO: Instruction Hierarchy via Constrained Reinforcement Learning [57.40686733111483]
  textscHIPO is a novel alignment framework that formulates HIF as a Constrained Markov Decision Process.<n>textscHIPO elevates system prompts from mere input context to strict algorithmic boundaries.
  arXiv  Detail & Related papers  (2026-03-17T06:12:41Z)
• Modular Prompt Optimization: Optimizing Structured Prompts with Section-Local Textual Gradients [0.8604557306886812]
  We introduce a schema-based prompt optimization framework that treats prompts as structured objects composed of fixed semantic sections.<n>We evaluate MPO on two reasoning benchmarks, ARC-Challenge and MMLU, using LLaMA-3 8B-Instruct and Mistral-7B-Instruct as solver models.
  arXiv  Detail & Related papers  (2026-01-07T16:20:08Z)
• Learning from Prompt itself: the Hierarchical Attribution Prompt Optimization [13.8868879878572]
  A structured optimization approach requires automated or semi-automated procedures to develop improved prompts.<n>Current prompt optimization methods often induce prompt drift, where new prompts fix prior failures but impair performance on previously successful tasks.<n>This study proposes the Hierarchical Prompt Optimization framework, which introduces three innovations: (1) a dynamic attribution mechanism targeting error patterns in training data and prompting history, (2) semantic-unit optimization for editing functional prompt segments, and (3) multimodal-friendly progression supporting both end-to-end LLM and LLM-MLLM.
  arXiv  Detail & Related papers  (2026-01-06T03:34:17Z)
• Relation-Aware Bayesian Optimization of DBMS Configurations Guided by Affinity Scores [2.474203056060563]
  Database Management Systems (DBMSs) are fundamental for managing large-scale and heterogeneous data, and their performance is critically influenced by configuration parameters.<n>Recent research has focused on automated configuration optimization using machine learning; however, existing approaches still exhibit several key limitations.<n>We propose RelTune, a novel framework that represents parameter dependencies as a Graph and learns GNN-based latent embeddings that encode performancerelevant semantics.
  arXiv  Detail & Related papers  (2025-10-31T03:46:42Z)
• VerIPO: Cultivating Long Reasoning in Video-LLMs via Verifier-Gudied Iterative Policy Optimization [59.39976343879587]
  VerIPO aims to gradually improve video LLMs' capacity for generating deep, long-term reasoning chains.<n>The training loop benefits from GRPO's expansive search and DPO's targeted optimization.<n>Our trained models exceed the direct inference of large-scale instruction-tuned Video-LLMs.
  arXiv  Detail & Related papers  (2025-05-25T06:41:28Z)
• On the Role of Feedback in Test-Time Scaling of Agentic AI Workflows [71.92083784393418]
  Agentic AI (systems that autonomously plan and act) are becoming widespread, yet their task success rate on complex tasks remains low.<n>Inference-time alignment relies on three components: sampling, evaluation, and feedback.<n>We introduce Iterative Agent Decoding (IAD), a procedure that repeatedly inserts feedback extracted from different forms of critiques.
  arXiv  Detail & Related papers  (2025-04-02T17:40:47Z)
• In-context Demonstration Matters: On Prompt Optimization for Pseudo-Supervision Refinement [71.60563181678323]
  Large language models (LLMs) have achieved great success across diverse tasks, and fine-tuning is sometimes needed to further enhance generation quality.<n>To handle these challenges, a direct solution is to generate high-confidence'' data from unsupervised downstream tasks.<n>We propose a novel approach, pseudo-supervised demonstrations aligned prompt optimization (PAPO) algorithm, which jointly refines both the prompt and the overall pseudo-supervision.
  arXiv  Detail & Related papers  (2024-10-04T03:39:28Z)
• Controllable Prompt Tuning For Balancing Group Distributional Robustness [53.336515056479705]
  We introduce an optimization scheme to achieve good performance across groups and find a good solution for all without severely sacrificing performance on any of them. We propose Controllable Prompt Tuning (CPT), which couples our approach with prompt-tuning techniques. On spurious correlation benchmarks, our procedures achieve state-of-the-art results across both transformer and non-transformer architectures, as well as unimodal and multimodal data.
  arXiv  Detail & Related papers  (2024-03-05T06:23:55Z)
• Robust Prompt Optimization for Large Language Models Against Distribution Shifts [80.6757997074956]
  Large Language Model (LLM) has demonstrated significant ability in various Natural Language Processing tasks. We propose a new problem of robust prompt optimization for LLMs against distribution shifts. This problem requires the prompt optimized over the labeled source group can simultaneously generalize to an unlabeled target group.
  arXiv  Detail & Related papers  (2023-05-23T11:30:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.