A Theoretical Analysis of Analogy-Based Evolutionary Transfer Optimization

• URL: http://arxiv.org/abs/2503.21156v1
• Date: Thu, 27 Mar 2025 04:49:20 GMT
• Title: A Theoretical Analysis of Analogy-Based Evolutionary Transfer Optimization
• Authors: Xiaoming Xue, Liang Feng, Yinglan Feng, Rui Liu, Kai Zhang, Kay Chen Tan,
• Abstract summary: We introduce analogical reasoning and link its subprocesses to three key issues in ETO.<n>We develop theories for analogy-based knowledge transfer rooted in the principles that underlie the subprocesses.<n>We present two theorems related to the performance gain of analogy-based knowledge transfer, namely unconditionally nonnegative performance gain and conditionally positive performance gain.
• Score: 22.185626881801234
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Evolutionary transfer optimization (ETO) has been gaining popularity in research over the years due to its outstanding knowledge transfer ability to address various challenges in optimization. However, a pressing issue in this field is that the invention of new ETO algorithms has far outpaced the development of fundamental theories needed to clearly understand the key factors contributing to the success of these algorithms for effective generalization. In response to this challenge, this study aims to establish theoretical foundations for analogy-based ETO, specifically to support various algorithms that frequently reference a key concept known as similarity. First, we introduce analogical reasoning and link its subprocesses to three key issues in ETO. Then, we develop theories for analogy-based knowledge transfer, rooted in the principles that underlie the subprocesses. Afterwards, we present two theorems related to the performance gain of analogy-based knowledge transfer, namely unconditionally nonnegative performance gain and conditionally positive performance gain, to theoretically demonstrate the effectiveness of various analogy-based ETO methods. Last but not least, we offer a novel insight into analogy-based ETO that interprets its conditional superiority over traditional evolutionary optimization through the lens of the no free lunch theorem for optimization.

Related papers

• Optimistic Algorithms for Adaptive Estimation of the Average Treatment Effect [36.25361703897723]
  Recent advances in martingale theory have paved the way for adaptive methods that can enhance the power of downstream inference. We study adaptive sampling procedures that take advantage of optimalally optimal causal inference procedures. Our findings mark a step forward in advancing adaptive causal inference methods in theory and practice.
  arXiv  Detail & Related papers  (2025-02-07T05:39:32Z)
• A Theoretical Review on Solving Algebra Problems [4.622321386568335]
  This paper first develops the State Transform Theory (STT), which emphasizes that the problem-solving algorithms are structured according to states and transforms.<n>This new construct accommodates the relation-centric algorithms for solving both word and diagrammatic algebra problems.
  arXiv  Detail & Related papers  (2024-10-29T08:16:49Z)
• See Further for Parameter Efficient Fine-tuning by Standing on the Shoulders of Decomposition [56.87609859444084]
  parameter-efficient fine-tuning (PEFT) focuses on optimizing a select subset of parameters while keeping the rest fixed, significantly lowering computational and storage overheads. We take the first step to unify all approaches by dissecting them from a decomposition perspective. We introduce two novel PEFT methods alongside a simple yet effective framework designed to enhance the performance of PEFT techniques across various applications.
  arXiv  Detail & Related papers  (2024-07-07T15:44:42Z)
• Bridging the Gap Between Theory and Practice: Benchmarking Transfer Evolutionary Optimization [31.603211545949414]
  This paper pioneers a practical TrEO benchmark suite, integrating problems from the literature categorized based on the three essential aspects of Big Source Task-Instances: volume, variety, and velocity. Our primary objective is to provide a comprehensive analysis of existing TrEO algorithms and pave the way for the development of new approaches to tackle practical challenges.
  arXiv  Detail & Related papers  (2024-04-20T13:34:46Z)
• Iterative Preference Learning from Human Feedback: Bridging Theory and Practice for RLHF under KL-Constraint [56.74058752955209]
  This paper studies the alignment process of generative models with Reinforcement Learning from Human Feedback (RLHF) We first identify the primary challenges of existing popular methods like offline PPO and offline DPO as lacking in strategical exploration of the environment. We propose efficient algorithms with finite-sample theoretical guarantees.
  arXiv  Detail & Related papers  (2023-12-18T18:58:42Z)
• Improving Energy Conserving Descent for Machine Learning: Theory and Practice [0.0]
  We develop the theory of Energy Con Descent (ECD) and introduce ECDSep, a gradient-based optimization algorithm able to tackle convex non-serving problems.
  arXiv  Detail & Related papers  (2023-06-01T05:15:34Z)
• Provable Reward-Agnostic Preference-Based Reinforcement Learning [61.39541986848391]
  Preference-based Reinforcement Learning (PbRL) is a paradigm in which an RL agent learns to optimize a task using pair-wise preference-based feedback over trajectories. We propose a theoretical reward-agnostic PbRL framework where exploratory trajectories that enable accurate learning of hidden reward functions are acquired.
  arXiv  Detail & Related papers  (2023-05-29T15:00:09Z)
• Hierarchical Optimization-Derived Learning [58.69200830655009]
  We establish a new framework, named Hierarchical ODL (HODL), to simultaneously investigate the intrinsic behaviors of optimization-derived model construction and its corresponding learning process. This is the first theoretical guarantee for these two coupled ODL components: optimization and learning.
  arXiv  Detail & Related papers  (2023-02-11T03:35:13Z)
• Towards KAB2S: Learning Key Knowledge from Single-Objective Problems to Multi-Objective Problem [8.049958323732362]
  ETO will overcome the traditional paradigm of zero reuse of related experience and knowledge from solved past problems in evolutionary computation. In scheduling applications via ETO, a quite appealing and highly competitive framework "meeting" could be formed for both intelligent scheduling and green scheduling.
  arXiv  Detail & Related papers  (2022-06-26T15:36:05Z)
• Towards Principled Disentanglement for Domain Generalization [90.9891372499545]
  A fundamental challenge for machine learning models is generalizing to out-of-distribution (OOD) data. We first formalize the OOD generalization problem as constrained optimization, called Disentanglement-constrained Domain Generalization (DDG) Based on the transformation, we propose a primal-dual algorithm for joint representation disentanglement and domain generalization.
  arXiv  Detail & Related papers  (2021-11-27T07:36:32Z)
• A Theoretical Framework for Target Propagation [75.52598682467817]
  We analyze target propagation (TP), a popular but not yet fully understood alternative to backpropagation (BP) Our theory shows that TP is closely related to Gauss-Newton optimization and thus substantially differs from BP. We provide a first solution to this problem through a novel reconstruction loss that improves feedback weight training.
  arXiv  Detail & Related papers  (2020-06-25T12:07:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.