Meta-Reinforcement Learning via Evolution for Multi-Objective Combinatorial Supply Chain Optimisation

• URL: http://arxiv.org/abs/2606.22146v1
• Date: Sat, 20 Jun 2026 16:55:48 GMT
• Title: Meta-Reinforcement Learning via Evolution for Multi-Objective Combinatorial Supply Chain Optimisation
• Abstract summary: We propose a population-based Meta-reinforcement learning framework that combines decomposition with evolutionary search in scalarisation weight space.<n>We evaluate the method in a multi-dimensional supply chain setting with conflicting economic, environmental, and social goals.
• Score: 3.2580935846865677
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Meta-reinforcement learning is a promising approach to multi-objective optimisation because it enables rapid policy adaptation across changing environments and preference settings. However, conventional few-shot methods usually fine-tune from a single shared meta-policy, which can reduce solution diversity and limit exploration of the Pareto front, especially in high-dimensional combinatorial problems such as supply chain optimisation. We propose a population-based Meta-reinforcement learning framework that combines decomposition with evolutionary search in scalarisation weight space. The framework maintains a population of weight vectors, each associated with a distinct meta-policy trained through gradient-based meta-learning, and iteratively refines this population through elitist selection, crossover, and mutation guided by hypervolume and entropy contributions. We evaluate the method in a multi-objective supply chain setting with conflicting economic, environmental, and social goals, and further test its generality on standard reinforcement learning problems. The results show that the proposed approach yields more diverse, better distributed Pareto front approximations, improves cross-task adaptation, increases hypervolume by up to 32\% over Meta-multi-objective reinforcement learning in the complex case, and attains the lowest average Hausdorff distance among all compared methods.

Related papers

• MGDA-Decoupled: Geometry-Aware Multi-Objective Optimisation for DPO-based LLM Alignment [6.301256425456381]
  We introduce MGDA-Decoupled, a geometry-based multi-objective optimisation algorithm.<n>It finds a shared descent direction while explicitly accounting for each objective's convergence dynamics.<n>Experiments on the UltraFeedback dataset show that MGDA-Decoupled achieves the highest win rates against golden responses.
  arXiv  Detail & Related papers  (2026-04-22T15:33:45Z)
• PCHC: Enabling Preference Conditioned Humanoid Control via Multi-Objective Reinforcement Learning [50.63196995993855]
  We propose a novel framework leveraging Multi-Objective Reinforcement Learning (MORL) to achieve Preference-Conditioned Humanoid Control (PCHC)<n>Our framework enables a single, preference-conditioned policy to exhibit a wide spectrum of diverse behaviors.
  arXiv  Detail & Related papers  (2026-03-25T07:55:37Z)
• PA2D-MORL: Pareto Ascent Directional Decomposition based Multi-Objective Reinforcement Learning [17.67661165622871]
  We propose the PA2D-MORL method, which constructs an efficient scheme for multi-objective problem decomposition and policy improvement.<n>We show that the proposed method clearly outperforms the current state-of-the-art algorithm in terms of both quality and stability of the outcomes.
  arXiv  Detail & Related papers  (2026-03-20T02:46:46Z)
• MIRACL: A Diverse Meta-Reinforcement Learning for Multi-Objective Multi-Echelon Combinatorial Supply Chain Optimisation [3.2580935846865677]
  We introduce MIRACL, a hierarchical Meta-MORL framework that allows for a few-shot generalisation across diverse tasks.<n>We show that MIRACL outperforms conventional MORL baselines in simple to moderate tasks, achieving up to 10% higher hypervolume and 5% better expected utility.
  arXiv  Detail & Related papers  (2026-03-05T23:42:26Z)
• SetPO: Set-Level Policy Optimization for Diversity-Preserving LLM Reasoning [50.93295951454092]
  We introduce a set level diversity objective defined over sampled trajectories using kernelized similarity.<n>Our approach derives a leave-one-out marginal contribution for each sampled trajectory and integrates this objective as a plug-in advantage shaping term for policy optimization.<n>Experiments across a range of model scales demonstrate the effectiveness of our proposed algorithm, consistently outperforming strong baselines in both Pass@1 and Pass@K across various benchmarks.
  arXiv  Detail & Related papers  (2026-02-01T07:13:20Z)
• Reinforcement Learning for Multi-Objective Multi-Echelon Supply Chain Optimisation [3.1194372040101928]
  The model is evaluated using a multi-objective reinforcement learning (RL) method, benchmarked against an originally single-objective RL algorithm modified with weighted sum.<n>We conduct experiments on varying network complexities, mimicking typical real-world challenges using a customisable simulator.<n>The model determines production and delivery quantities across supply chain routes to achieve near-optimal trade-offs between competing objectives.
  arXiv  Detail & Related papers  (2025-07-26T04:30:11Z)
• HarmoDT: Harmony Multi-Task Decision Transformer for Offline Reinforcement Learning [72.25707314772254]
  We introduce the Harmony Multi-Task Decision Transformer (HarmoDT), a novel solution designed to identify an optimal harmony subspace of parameters for each task. The upper level of this framework is dedicated to learning a task-specific mask that delineates the harmony subspace, while the inner level focuses on updating parameters to enhance the overall performance of the unified policy.
  arXiv  Detail & Related papers  (2024-05-28T11:41:41Z)
• UCB-driven Utility Function Search for Multi-objective Reinforcement Learning [51.00436121587591]
  In Multi-objective Reinforcement Learning (MORL) agents are tasked with optimising decision-making behaviours.<n>We focus on the case of linear utility functions parametrised by weight vectors w.<n>We introduce a method based on Upper Confidence Bound to efficiently search for the most promising weight vectors during different stages of the learning process.
  arXiv  Detail & Related papers  (2024-05-01T09:34:42Z)
• Latent-Conditioned Policy Gradient for Multi-Objective Deep Reinforcement Learning [2.1408617023874443]
  We propose a novel multi-objective reinforcement learning (MORL) algorithm that trains a single neural network via policy gradient. The proposed method works in both continuous and discrete action spaces with no design change of the policy network.
  arXiv  Detail & Related papers  (2023-03-15T20:07:48Z)
• Faster Last-iterate Convergence of Policy Optimization in Zero-Sum Markov Games [63.60117916422867]
  This paper focuses on the most basic setting of competitive multi-agent RL, namely two-player zero-sum Markov games. We propose a single-loop policy optimization method with symmetric updates from both agents, where the policy is updated via the entropy-regularized optimistic multiplicative weights update (OMWU) method. Our convergence results improve upon the best known complexities, and lead to a better understanding of policy optimization in competitive Markov games.
  arXiv  Detail & Related papers  (2022-10-03T16:05:43Z)
• Permutation Invariant Policy Optimization for Mean-Field Multi-Agent Reinforcement Learning: A Principled Approach [128.62787284435007]
  We propose the mean-field proximal policy optimization (MF-PPO) algorithm, at the core of which is a permutation-invariant actor-critic neural architecture. We prove that MF-PPO attains the globally optimal policy at a sublinear rate of convergence. In particular, we show that the inductive bias introduced by the permutation-invariant neural architecture enables MF-PPO to outperform existing competitors.
  arXiv  Detail & Related papers  (2021-05-18T04:35:41Z)
• Large-Scale Meta-Learning with Continual Trajectory Shifting [76.29017270864308]
  We show that allowing the meta-learners to take a larger number of inner gradient steps better captures the structure of heterogeneous and large-scale tasks. In order to increase the frequency of meta-updates, we propose to estimate the required shift of the task-specific parameters. We show that the algorithm largely outperforms the previous first-order meta-learning methods in terms of both generalization performance and convergence.
  arXiv  Detail & Related papers  (2021-02-14T18:36:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.