Automated Metaheuristic Algorithm Design with Autoregressive Learning

• URL: http://arxiv.org/abs/2405.03419v1
• Date: Mon, 6 May 2024 12:36:17 GMT
• Title: Automated Metaheuristic Algorithm Design with Autoregressive Learning
• Authors: Qi Zhao, Tengfei Liu, Bai Yan, Qiqi Duan, Jian Yang, Yuhui Shi,
• Abstract summary: Current automated methods design algorithms within a fixed structure and operate from scratch. This paper proposes an autoregressive learning-based designer for automated design of metaheuristic algorithms.
• Score: 25.967262411437403
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automated design of metaheuristic algorithms offers an attractive avenue to reduce human effort and gain enhanced performance beyond human intuition. Current automated methods design algorithms within a fixed structure and operate from scratch. This poses a clear gap towards fully discovering potentials over the metaheuristic family and fertilizing from prior design experience. To bridge the gap, this paper proposes an autoregressive learning-based designer for automated design of metaheuristic algorithms. Our designer formulates metaheuristic algorithm design as a sequence generation task, and harnesses an autoregressive generative network to handle the task. This offers two advances. First, through autoregressive inference, the designer generates algorithms with diverse lengths and structures, enabling to fully discover potentials over the metaheuristic family. Second, prior design knowledge learned and accumulated in neurons of the designer can be retrieved for designing algorithms for future problems, paving the way to continual design of algorithms for open-ended problem-solving. Extensive experiments on numeral benchmarks and real-world problems reveal that the proposed designer generates algorithms that outperform all human-created baselines on 24 out of 25 test problems. The generated algorithms display various structures and behaviors, reasonably fitting for different problem-solving contexts. Code will be released after paper publication.

Related papers

• A Generalized Evolutionary Metaheuristic (GEM) Algorithm for Engineering Optimization [1.6589012298747952]
  A major trend in recent years is the use of nature-inspired metaheustic algorithms (NIMA) There are over 540 algorithms in the literature, and there is no unified framework to understand the search mechanisms of different algorithms. We propose a generalized evolutionary metaheuristic algorithm to unify more than 20 different algorithms.
  arXiv  Detail & Related papers  (2024-07-02T09:55:15Z)
• Contractual Reinforcement Learning: Pulling Arms with Invisible Hands [68.77645200579181]
  We propose a theoretical framework for aligning economic interests of different stakeholders in the online learning problems through contract design. For the planning problem, we design an efficient dynamic programming algorithm to determine the optimal contracts against the far-sighted agent. For the learning problem, we introduce a generic design of no-regret learning algorithms to untangle the challenges from robust design of contracts to the balance of exploration and exploitation.
  arXiv  Detail & Related papers  (2024-07-01T16:53:00Z)
• Automated Design of Metaheuristic Algorithms: A Survey [16.5686507795359]
  This paper presents a broad picture of automated design of metaheuristic algorithms. With computing power to fully explore potential design choices, the automated design could reach and even surpass human-level design.
  arXiv  Detail & Related papers  (2023-03-12T01:20:49Z)
• Investigating Positive and Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniques [55.492211642128446]
  Designers reportedly struggle with design optimization tasks where they are asked to find a combination of design parameters that maximizes a given set of objectives. Model-based computational design algorithms assist designers by generating design examples during design. Black box methods for assistance, on the other hand, can work with any design problem.
  arXiv  Detail & Related papers  (2022-04-15T20:40:43Z)
• Automated Design of Salient Object Detection Algorithms with Brain Programming [3.518016233072556]
  This research work proposes expanding the artificial dorsal stream using a recent proposal to solve salient object detection problems. We decided to apply the fusion of visual saliency and image segmentation algorithms as a template. We present results on a benchmark designed by experts with outstanding results in comparison with the state-of-the-art.
  arXiv  Detail & Related papers  (2022-04-07T20:21:30Z)
• AutoOpt: A General Framework for Automatically Designing Metaheuristic Optimization Algorithms with Diverse Structures [15.140309031326076]
  This paper proposes a general framework, AutoOpt, for automatically designing metaheuristic algorithms with diverse structures. A general algorithm prototype dedicated to covering the metaheuristic family as widely as possible. A directed acyclic graph algorithm representation to fit the proposed prototype. A graph representation embedding method offering an alternative compact form of the graph to be manipulated.
  arXiv  Detail & Related papers  (2022-04-03T05:31:56Z)
• AutoSpace: Neural Architecture Search with Less Human Interference [84.42680793945007]
  Current neural architecture search (NAS) algorithms still require expert knowledge and effort to design a search space for network construction. We propose a novel differentiable evolutionary framework named AutoSpace, which evolves the search space to an optimal one. With the learned search space, the performance of recent NAS algorithms can be improved significantly compared with using previously manually designed spaces.
  arXiv  Detail & Related papers  (2021-03-22T13:28:56Z)
• Evolving Reinforcement Learning Algorithms [186.62294652057062]
  We propose a method for meta-learning reinforcement learning algorithms. The learned algorithms are domain-agnostic and can generalize to new environments not seen during training. We highlight two learned algorithms which obtain good generalization performance over other classical control tasks, gridworld type tasks, and Atari games.
  arXiv  Detail & Related papers  (2021-01-08T18:55:07Z)
• It's the Journey Not the Destination: Building Genetic Algorithms Practitioners Can Trust [0.0]
  Survey shows attitudes of engineers and students with design experience with respect to optimisation algorithms. A common thread throughout participants responses is that there is a question of trust towards genetic algorithms within industry. Participants have expressed a desire to continue to remain in the design loop.
  arXiv  Detail & Related papers  (2020-10-13T14:07:30Z)
• Learning to Stop While Learning to Predict [85.7136203122784]
  Many algorithm-inspired deep models are restricted to a fixed-depth'' for all inputs. Similar to algorithms, the optimal depth of a deep architecture may be different for different input instances. In this paper, we tackle this varying depth problem using a steerable architecture. We show that the learned deep model along with the stopping policy improves the performances on a diverse set of tasks.
  arXiv  Detail & Related papers  (2020-06-09T07:22:01Z)
• AutoML-Zero: Evolving Machine Learning Algorithms From Scratch [76.83052807776276]
  We show that it is possible to automatically discover complete machine learning algorithms just using basic mathematical operations as building blocks. We demonstrate this by introducing a novel framework that significantly reduces human bias through a generic search space. We believe these preliminary successes in discovering machine learning algorithms from scratch indicate a promising new direction in the field.
  arXiv  Detail & Related papers  (2020-03-06T19:00:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.