Teacher-Aware Evolution of Heuristic Programs from Learned Optimization Policies

• URL: http://arxiv.org/abs/2605.10634v1
• Date: Mon, 11 May 2026 14:22:29 GMT
• Title: Teacher-Aware Evolution of Heuristic Programs from Learned Optimization Policies
• Abstract summary: We propose a emph-teacher-aware evolutionary framework that uses independently trained optimization policies as behavioral teachers.<n>Instead of deploying or imitating the teacher, our method queries it on states visited by candidate programs and uses its action preferences as local feedback for evolution.
• Score: 2.979229299806738
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLM-based automatic heuristic design has shown promise for generating executable heuristics for combinatorial optimization, but existing methods mainly rely on delayed endpoint performance. We propose a \emph{teacher-aware evolutionary framework} that uses independently trained learned optimization policies as behavioral teachers. Instead of deploying or imitating the teacher, our method queries it on states visited by candidate heuristic programs and uses its action preferences as local feedback for evolution. The resulting search discovers static executable heuristics guided by both task performance and teacher-derived behavioral signals. Experiments on scheduling, routing, and graph optimization benchmarks show that our method improves over performance-driven LLM heuristic evolution baselines while requiring no neural inference at deployment. These results suggest that learned optimization policies can be repurposed as behavioral feedback sources for automatic heuristic discovery.

Related papers

• TAPO: Transition-Aware Policy Optimization for LLM Agents [27.95488836373643]
  TAPO is a unified training framework that alternates between policy optimization and transition supervision.<n>It serves as a computationally lightweight, plug-and-play enhancement module for existing agent RL algorithms.<n>We conduct systematic experiments on WebShop and ALFWorld, integrating foundation models of various scales with different policy optimization algorithms.
  arXiv  Detail & Related papers  (2026-07-30T10:17:55Z)
• PAWS: Preference Learning with Advantage-Weighted Segments [61.590521943273]
  We propose a segment-based preference learning method that performs policy updates directly using segment-level advantage functions.<n>Experiments on simulated robotic manipulation and locomotion tasks demonstrate that PAWS consistently outperforms existing PbRL approaches.
  arXiv  Detail & Related papers  (2026-06-10T12:00:17Z)
• Evolution of Optimization Methods: Algorithms, Scenarios, and Evaluations [98.44542103979735]
  First-order gradient descent methods, epitomized by gradient descent (SGD) and Adam, serve as the cornerstone of modern training pipelines.<n>Large-scale model training, stringent privacy requirements, and distributed learning paradigms expose critical limitations in these conventional approaches regarding privacy protection and memory efficiency.<n>We retrospectively analyze the evolutionary trajectory of deep learning optimization algorithms and present a comprehensive empirical evaluation of mainstreams across diverse model architectures and training scenarios.<n>We distill key emerging trends and fundamental design trade-offs, pinpointing promising directions for future research.
  arXiv  Detail & Related papers  (2026-04-14T17:01:36Z)
• Learning to Learn-at-Test-Time: Language Agents with Learnable Adaptation Policies [48.31565681022125]
  Test-Time Learning (TTL) enables language agents to iteratively refine their performance through repeated interactions with the environment at inference time.<n>We argue that optimal adaptation policies should be learned from task environments, not hand-engineered based on human intuition.<n>We introduce Meta-TTL, a framework that formulates the discovery of effective adaptation policies as a bi-level optimization problem.
  arXiv  Detail & Related papers  (2026-04-01T12:41:01Z)
• Agentic Policy Optimization via Instruction-Policy Co-Evolution [44.74237684380034]
  INSPO is a novel framework for instruction-policy co-evolution.<n>It integrates instruction optimization as a dynamic component of the reinforcement learning loop.<n>In experiments, INSPO achieves substantial performance gains with only a marginal increase in computational overhead.
  arXiv  Detail & Related papers  (2025-12-01T17:56:29Z)
• Discovering Preference Optimization Algorithms with and for Large Language Models [50.843710797024805]
  offline preference optimization is a key method for enhancing and controlling the quality of Large Language Model (LLM) outputs. We perform objective discovery to automatically discover new state-of-the-art preference optimization algorithms without (expert) human intervention. Experiments demonstrate the state-of-the-art performance of DiscoPOP, a novel algorithm that adaptively blends logistic and exponential losses.
  arXiv  Detail & Related papers  (2024-06-12T16:58:41Z)
• Unleashing the Potential of Large Language Models as Prompt Optimizers: Analogical Analysis with Gradient-based Model Optimizers [108.72225067368592]
  We propose a novel perspective to investigate the design of large language models (LLMs)-based prompts.<n>We identify two pivotal factors in model parameter learning: update direction and update method.<n>We develop a capable Gradient-inspired Prompt-based GPO.
  arXiv  Detail & Related papers  (2024-02-27T15:05:32Z)
• Acceleration in Policy Optimization [50.323182853069184]
  We work towards a unifying paradigm for accelerating policy optimization methods in reinforcement learning (RL) by integrating foresight in the policy improvement step via optimistic and adaptive updates. We define optimism as predictive modelling of the future behavior of a policy, and adaptivity as taking immediate and anticipatory corrective actions to mitigate errors from overshooting predictions or delayed responses to change. We design an optimistic policy gradient algorithm, adaptive via meta-gradient learning, and empirically highlight several design choices pertaining to acceleration, in an illustrative task.
  arXiv  Detail & Related papers  (2023-06-18T15:50:57Z)
• A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases [44.01339030872185]
  Blackbox learneds often struggle with stability and generalization when applied to tasks unlike those in their meta-training set. We investigate the inductive biases and stability properties of optimization algorithms, and apply the resulting insights to designing inductive biases for blackboxs. We learn to a variety of neural network training tasks, where it outperforms the current state of the art learned.
  arXiv  Detail & Related papers  (2022-09-22T17:47:21Z)
• Reverse engineering learned optimizers reveals known and novel mechanisms [50.50540910474342]
  Learneds are algorithms that can themselves be trained to solve optimization problems. Our results help elucidate the previously murky understanding of how learneds work, and establish tools for interpreting future learneds.
  arXiv  Detail & Related papers  (2020-11-04T07:12:43Z)
• Value Driven Representation for Human-in-the-Loop Reinforcement Learning [33.79501890330252]
  We focus on algorithmic foundation of how to help the system designer choose the set of sensors or features to define the observation space used by reinforcement learning agent. We present an algorithm, value driven representation (VDR) that can iteratively and adaptively augment the observation space of a reinforcement learning agent. We evaluate the performance of our approach on standard RL benchmarks with simulated humans and demonstrate significant improvement over prior baselines.
  arXiv  Detail & Related papers  (2020-04-02T18:45:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.