MOSCOPT: Mixture-of-Skills Collective Optimization for LLM Agents

• URL: http://arxiv.org/abs/2609.14399v1
• Date: Sun, 13 Sep 2026 09:57:59 GMT
• Title: MOSCOPT: Mixture-of-Skills Collective Optimization for LLM Agents
• Abstract summary: MOSCOPT is a text-native, parameter-free algorithm that jointly optimize a pool of $N$ skills and a gating skill $G$ that dynamically selects $K$ skills per step.<n>Experiments and detailed ablations across 5 benchmarks and 3 target LLMs demonstrate that MOSCOPT consistently outperforms all baselines.
• Score: 0.9883261192383612
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Natural language prompts and skills serve as the strategic backbone of LLM-based agents. Recent advances in prompt and skill optimization have achieved notable gains, yet all existing methods optimize a \emph{single} text template---missing the synergy among multiple complementary strategies. We propose MOSCOPT, a text-native, parameter-free algorithm that jointly optimizes a pool of $N$ skills and a gating skill $G$ that dynamically selects $K$ skills per step. To effectively optimize the skills, we build the EditAdam with internally maintained dual states. Through the three-phase interleaved updates with EditAdam, the system monotonically improves without gradient or parameter tuning. Extensive experiments and detailed ablations across 5 benchmarks and 3 target LLMs demonstrate that MOSCOPT consistently outperforms all baselines, and confirm that both the mixture-of-skills architecture with selective activation and the collective evolution with three-phase interleaving are essential to its superior performance. Code is released https://github.com/zhangzhenyu13/SummerClaw/tree/master/summerclaw/agent_trainer/algorithms/moscopt.

Related papers

• A Hybrid Nested Harness for Decoupling Structure and Parameters in LLM-Driven Optimization [6.599344783327054]
  In evolutionary algorithms powered by language models, the LLM acts as a single operator that simultaneously updates structural components and continuous parameters.<n>We formalize a hybrid nested search, in which an outer loop has the LLM propose a structural sketch, with numeric gaps, and an innernest tunes the sketch.<n>We validate our framework across three scientific domains: (i) meta-optimizers on closed-form test functions, (ii) code-based policies for systems research and social dilemmas, and (iii) approximate Bayesian inference tasks.
  arXiv  Detail & Related papers  (2026-08-08T14:35:22Z)
• $\mathcal{S}^2$IT: Stepwise Syntax Integration Tuning for Large Language Models in Aspect Sentiment Quad Prediction [51.165432266846096]
  syntactic structure information is underutilized in the generative paradigm of large language models (LLMs)<n>S2IT progressively integrates syntactic structure knowledge into LLMs through a multi-step tuning process.<n> Experiments demonstrate that S2IT significantly improves state-of-the-art performance across multiple datasets.
  arXiv  Detail & Related papers  (2026-04-25T13:24:29Z)
• SkillMOO: Multi-Objective Optimization of Agent Skills for Software Engineering [9.008257157319271]
  SkillMOO improves pass rate by up to 131% while reducing cost up to 32% relative to the best baseline per task at low optimization overhead.<n>We present SkillMOO, a multi-objective optimization framework that automatically evolves skill bundles using LLM-proposed edits and NSGA-II survivor selection.<n>On three SkillsBench software engineering tasks, SkillMOO improves pass rate by up to 131% while reducing cost up to 32% relative to the best baseline per task at low optimization overhead.
  arXiv  Detail & Related papers  (2026-04-10T13:08:01Z)
• MetaClaw: Just Talk -- An Agent That Meta-Learns and Evolves in the Wild [74.7263562191605]
  Large language model (LLM) agents are increasingly used for complex tasks.<n>Existing methods either store raw trajectories without distilling knowledge, maintain static skill libraries, or require disruptive downtime for retraining.<n>We present MetaClaw, a continual meta-learning framework that jointly evolves a base LLM policy and a library of reusable behavioral skills.
  arXiv  Detail & Related papers  (2026-03-17T22:30:30Z)
• Taming LLMs by Scaling Learning Rates with Gradient Grouping [49.91587150497186]
  Training large language models (LLMs) poses challenges due to their massive scale and heterogeneous architectures.<n>This work introduces Scaling with Gradient Grouping (SGG), an gradient wrapper that improves adaptive learning rate estimation by dynamic grouping and group-specific scaling.
  arXiv  Detail & Related papers  (2025-06-01T15:30:37Z)
• AlphaAdam:Asynchronous Masked Optimization with Dynamic Alpha for Selective Updates [17.490809667438818]
  We propose AlphaAdam, an optimization framework for large language models (LLMs)<n>By decoupling parameter updates and dynamically adjusting their strength, AlphaAdam accelerates convergence and improves training stability.
  arXiv  Detail & Related papers  (2025-01-30T02:10:23Z)
• Optima: Optimizing Effectiveness and Efficiency for LLM-Based Multi-Agent System [75.25394449773052]
  Large Language Model (LLM) based multi-agent systems (MAS) show remarkable potential in collaborative problem-solving.<n>Yet they still face critical challenges: low communication efficiency, poor scalability, and a lack of effective parameter-updating optimization methods.<n>We present Optima, a novel framework that addresses these issues by significantly enhancing both communication efficiency and task effectiveness.
  arXiv  Detail & Related papers  (2024-10-10T17:00:06Z)
• Decoding-Time Language Model Alignment with Multiple Objectives [116.42095026960598]
  Existing methods primarily focus on optimizing LMs for a single reward function, limiting their adaptability to varied objectives. Here, we propose $textbfmulti-objective decoding (MOD)$, a decoding-time algorithm that outputs the next token from a linear combination of predictions. We show why existing approaches can be sub-optimal even in natural settings and obtain optimality guarantees for our method.
  arXiv  Detail & Related papers  (2024-06-27T02:46:30Z)
• LLM as a Complementary Optimizer to Gradient Descent: A Case Study in Prompt Tuning [69.95292905263393]
  We show that gradient-based and high-level LLMs can effectively collaborate a combined optimization framework.<n>In this paper, we show that these complementary to each other and can effectively collaborate a combined optimization framework.
  arXiv  Detail & Related papers  (2024-05-30T06:24:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.