Evolutionary Soups: Evolving Mixture-of-Experts for Multi-Objective LLM Alignment

• URL: http://arxiv.org/abs/2608.29978v1
• Date: Sun, 30 Aug 2026 19:09:59 GMT
• Title: Evolutionary Soups: Evolving Mixture-of-Experts for Multi-Objective LLM Alignment
• Abstract summary: Large language models are increasingly required to generate responses that satisfy multiple competing objectives.<n>We propose Evolutionary Soups, a framework for fine-grained generation using gating networks trained via evolutionary algorithm.<n> Experiments across three tasks demonstrate the effectiveness of Evolutionary Soups over baselines.
• Score: 14.899821854922434
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Large language models are increasingly required to generate responses that satisfy multiple competing objectives. Since optimal trade-offs depend on both user preferences and input prompts, controllable multi-objective generation must dynamically adapt models at inference time without retraining. To address this, we propose Evolutionary Soups, a mixture-of-experts framework for fine-grained generation control, with gating networks trained via an evolutionary algorithm. The per-layer gating networks dynamically produce expert-merging coefficients from hidden-state representations, while the evolutionary algorithm incorporates greedy hypervolume contribution for effective evolution of these gating networks, achieving consistent improvements on large and noisy training datasets and broader coverage of the non-convex Pareto front. Experiments across three tasks demonstrate the effectiveness of Evolutionary Soups over baselines: it achieves the best hypervolume, linear utility, and Tchebyshev utility (~20% improvement) among controllable methods on all tasks.

Related papers

• Self-Evolving Memory for Generative Recommendation [100.15207958893029]
  Generative recommendation has emerged as a promising end-to-end paradigm for personalized recommendation.<n>Existing evolving strategies, such as continual retraining and distillation-based adaptation, directly update the shared model parameters.<n>We propose a self-evolving memory paradigm for generative recommendation, aiming to enable effective evolution across heterogeneous behavioral patterns.
  arXiv  Detail & Related papers  (2026-09-14T14:00:38Z)
• GALA: Generative Aligned Learning for Adaptive Multimodal Representation in the Taobao Shangou Recommender System [4.96441528622156]
  We present GALA, a three-stage pipeline that constructs multimodal pretraining data from user behavior and refines it via conversion-based rewards.<n>GALA has been deployed in the production environment at Taobao Shangou, serving over 200 million daily active users.<n>It delivers consistent offline gains of +0.12/+0.20 AUC along with better PCOC metrics.
  arXiv  Detail & Related papers  (2026-07-31T09:32:01Z)
• Co-Evolved Spiking Neural Network Ensembles via Marginal Contribution Fitness [0.0]
  Co-evolutionary ensembles evolve with fitness defined by each network's marginal contribution to group performance.<n>We evaluate the approach on classification, regression, and control tasks under $$Caspian neuromorphic hardware constraints.
  arXiv  Detail & Related papers  (2026-06-12T00:04:06Z)
• EvoSelect: Data-Efficient LLM Evolution for Targeted Task Adaptation [79.71802168256542]
  Adapting large language models (LLMs) to a targeted task efficiently and effectively remains a fundamental challenge.<n>One straightforward approach is through an iterative generation-training loop, where candidate data are synthesized through an external generator.<n>We introduce a refined paradigm, namely an iterative generation-selection-training loop, which incorporates a selection step prior to model updates.
  arXiv  Detail & Related papers  (2026-04-28T23:26:16Z)
• Neural Network Optimization Reimagined: Decoupled Techniques for Scratch and Fine-Tuning [49.751529745537546]
  We propose DualOpt, a novel approach that decouples optimization techniques specifically tailored for training from scratch.<n>For training from scratch, we introduce real-time layer-wise weight decay, designed to enhance both convergence and generalization.<n>We extend the layer-wise weight decay to dynamically adjust the rollback levels across layers, adapting to the varying demands of different downstream tasks.
  arXiv  Detail & Related papers  (2026-04-21T06:27:18Z)
• A Hierarchical Importance-Guided Multi-objective Evolutionary Framework for Deep Neural Network Pruning [0.0]
  This paper introduces a hierarchical importance objective framework that reformulates a tractable large-scale multi-objective optimization problem.<n>The proposed method contributes to a scalable evolutionary approach for solving very-large-scale multi-objective optimization problems.
  arXiv  Detail & Related papers  (2026-04-01T16:08:42Z)
• AdaEvolve: Adaptive LLM Driven Zeroth-Order Optimization [61.535567824938205]
  We introduce AdaEvolve, a framework that reformulates LLM-driven evolution as a hierarchical adaptive optimization problem.<n>AdaEvolve consistently outperforms the open-ended baselines across 185 different open-ended optimization problems.
  arXiv  Detail & Related papers  (2026-02-23T18:45:31Z)
• OmniVL-Guard: Towards Unified Vision-Language Forgery Detection and Grounding via Balanced RL [63.388513841293616]
  Existing forgery detection methods fail to handle the interleaved text, images, and videos prevalent in real-world misinformation.<n>To bridge this gap, this paper targets to develop a unified framework for omnibus vision-language forgery detection and grounding.<n>We propose textbf OmniVL-Guard, a balanced reinforcement learning framework for omnibus vision-language forgery detection and grounding.
  arXiv  Detail & Related papers  (2026-02-11T09:41:36Z)
• Towards Robust Universal Perturbation Attacks: A Float-Coded, Penalty-Driven Evolutionary Approach [10.211523683826004]
  Universal adversarial noise perturbations (UAPs) have garnered significant attention due to their ability to undermine deep neural networks.<n>We introduce a single-objective-driven framework for generating such perturbations.<n>Our framework consistently produces perturbations with smaller sizes, higher misclassification effectiveness, and faster compared to existing methods.
  arXiv  Detail & Related papers  (2026-01-18T23:31:12Z)
• Controlled Self-Evolution for Algorithmic Code Optimization [33.82967000330864]
  Self-evolution methods enhance code generation through iterative "generate-verify-refine" cycles.<n>Existing approaches fail to discover solutions with superior complexity within limited budgets.<n>We propose Controlled Self-Evolution (CSE), which consists of three key components.
  arXiv  Detail & Related papers  (2026-01-12T09:23:13Z)
• Improving Deepfake Detection with Reinforcement Learning-Based Adaptive Data Augmentation [60.04281435591454]
  CRDA (Curriculum Reinforcement-Learning Data Augmentation) is a novel framework guiding detectors to progressively master multi-domain forgery features.<n>Central to our approach is integrating reinforcement learning and causal inference.<n>Our method significantly improves detector generalizability, outperforming SOTA methods across multiple cross-domain datasets.
  arXiv  Detail & Related papers  (2025-11-10T12:45:52Z)
• Multi-Agent Collaboration via Evolving Orchestration [55.574417128944226]
  Large language models (LLMs) have achieved remarkable results across diverse downstream tasks, but their monolithic nature restricts scalability and efficiency in complex problem-solving.<n>We propose a puppeteer-style paradigm for LLM-based multi-agent collaboration, where a centralized orchestrator ("puppeteer") dynamically directs agents ("puppets") in response to evolving task states.<n> Experiments on closed- and open-domain scenarios show that this method achieves superior performance with reduced computational costs.
  arXiv  Detail & Related papers  (2025-05-26T07:02:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.