AgentPSO: Evolving Agent Reasoning Skill via Multi-agent Particle Swarm Optimization

• URL: http://arxiv.org/abs/2605.08704v1
• Date: Sat, 09 May 2026 05:38:21 GMT
• Title: AgentPSO: Evolving Agent Reasoning Skill via Multi-agent Particle Swarm Optimization
• Abstract summary: We introduce AgentPSO, a framework for evolving multi-agent reasoning skills.<n>AgentPSO treats each agent as a particle-like reasoner whose state is a natural-language skill.<n>It iteratively moves agents toward stronger skill states to improve both individual and collective reasoning performance.
• Score: 46.78426363253909
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multi-agent reasoning has shown promise for improving the problem-solving ability of large language models by allowing multiple agents to explore diverse reasoning paths. However, most existing multi-agent methods rely on inference-time debate or aggregation, which can be vulnerable to incorrect peer influence and biased consensus. Moreover, the agents themselves remain static, as their underlying reasoning skills do not evolve across tasks. In this paper, we introduce AgentPSO, a particle-swarm-inspired framework for evolving multi-agent reasoning skills. AgentPSO treats each agent as a particle-like reasoner whose state is a natural-language skill and whose velocity is a semantic update direction, iteratively moving agents toward stronger skill states to improve both individual and collective reasoning performance. Across training iterations, each agent updates its skill by combining its previous velocity, personal-best skill, global-best skill, and a self-reflective direction derived from peer reasoning trajectories. This enables agents to learn reusable reasoning behaviors from both their own experiences and the strongest skills discovered by the population, without updating the parameters of the backbone language model. Experiments on mathematical and general reasoning benchmarks show that AgentPSO improves over static single-agent skills and test-time-only multi-agent reasoning baselines. The evolved skills further transfer across benchmarks and to another backbone model, suggesting that AgentPSO captures reusable reasoning procedures rather than merely optimizing benchmark-specific prompts. Code is open-sourced at https://github.com/HYUNMIN-HWANG/AgentPSO/.

Related papers

• AgentFugue: Agent Scaling for Long-Horizon Tasks through Collective Reasoning [57.72181998599248]
  AgentFugue is a collective reasoning framework built around a shared reasoning hub.<n>As peer agents explore the same task in parallel, the hub records concise notes on what each agent has established, attempted, or ruled out.<n>Our results suggest that collective reasoning can turn scaling out peer agent systems into a distinct source of capability gains.
  arXiv  Detail & Related papers  (2026-05-23T09:21:51Z)
• EVOCHAMBER: Test-Time Co-evolution of Multi-Agent System at Individual, Team, and Population Scales [27.568287645700064]
  A single-agent learner can only evolve its own context and memory.<n>A multi-agent system additionally evolves who collaborates, how they collaborate, and how knowledge flows across the population.<n>We present EVOCHAMBER, a training-free framework that instantiates test-time evolution at three levels over a coevolving agent pool.
  arXiv  Detail & Related papers  (2026-05-11T18:42:33Z)
• AgentArk: Distilling Multi-Agent Intelligence into a Single LLM Agent [57.10083973844841]
  AgentArk is a novel framework to distill multi-agent dynamics into the weights of a single model.<n>We investigate three hierarchical distillation strategies across various models, tasks, scaling, and scenarios.<n>By shifting the burden of computation from inference to training, the distilled models preserve the efficiency of one agent while exhibiting strong reasoning and self-correction performance of multiple agents.
  arXiv  Detail & Related papers  (2026-02-03T19:18:28Z)
• OPTAGENT: Optimizing Multi-Agent LLM Interactions Through Verbal Reinforcement Learning for Enhanced Reasoning [14.105640933123325]
  Large Language Models (LLMs) have shown remarkable reasoning capabilities in mathematical and scientific tasks.<n>To enhance complex reasoning, multi-agent systems have been proposed to harness the collective intelligence of LLM agents.<n>We propose $ours$, a multi-agent verbal reinforcement learning algorithm that dynamically constructs and refines multi-agent collaboration structures.
  arXiv  Detail & Related papers  (2025-10-20T19:07:51Z)
• Rethinking Agent Design: From Top-Down Workflows to Bottom-Up Skill Evolution [34.66260172204154]
  We introduce a bottom-up agent paradigm that mirrors the human learning process.<n>Agents acquire competence through a trial-and-reasoning mechanism-exploring, reflecting on outcomes, and abstracting skills over time.<n>We evaluate this paradigm in Slay the Spire and Civilization V, where agents perceive through raw visual inputs and act via mouse outputs, the same as human players.
  arXiv  Detail & Related papers  (2025-05-23T09:38:55Z)
• OpenWebVoyager: Building Multimodal Web Agents via Iterative Real-World Exploration, Feedback and Optimization [66.22117723598872]
  We introduce an open-source framework designed to facilitate the development of multimodal web agent. We first train the base model with imitation learning to gain the basic abilities. We then let the agent explore the open web and collect feedback on its trajectories.
  arXiv  Detail & Related papers  (2024-10-25T15:01:27Z)
• Gödel Agent: A Self-Referential Agent Framework for Recursive Self-Improvement [112.04307762405669]
  G"odel Agent is a self-evolving framework inspired by the G"odel machine.<n>G"odel Agent can achieve continuous self-improvement, surpassing manually crafted agents in performance, efficiency, and generalizability.
  arXiv  Detail & Related papers  (2024-10-06T10:49:40Z)
• AgentGym: Evolving Large Language Model-based Agents across Diverse Environments [116.97648507802926]
  Large language models (LLMs) are considered a promising foundation to build such agents. We take the first step towards building generally-capable LLM-based agents with self-evolution ability. We propose AgentGym, a new framework featuring a variety of environments and tasks for broad, real-time, uni-format, and concurrent agent exploration.
  arXiv  Detail & Related papers  (2024-06-06T15:15:41Z)
• Skill Discovery of Coordination in Multi-agent Reinforcement Learning [41.67943127631515]
  We propose "Multi-agent Skill Discovery"(MASD), a method for discovering skills for coordination patterns of multiple agents. We show the emergence of various skills on the level of coordination in a general particle multi-agent environment. We also reveal that the "bottleneck" prevents skills from collapsing to a single agent and enhances the diversity of learned skills.
  arXiv  Detail & Related papers  (2020-06-07T02:04:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.