LangMARL: Natural Language Multi-Agent Reinforcement Learning

• URL: http://arxiv.org/abs/2604.00722v1
• Date: Wed, 01 Apr 2026 10:32:00 GMT
• Title: LangMARL: Natural Language Multi-Agent Reinforcement Learning
• Abstract summary: Large language model (LLM) agents struggle to autonomously evolve coordination strategies in dynamic environments.<n>We propose LangMARL, a framework that brings credit assignment and policy gradient evolution from cooperative MARL into the language space.
• Score: 36.24443050447855
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language model (LLM) agents struggle to autonomously evolve coordination strategies in dynamic environments, largely because coarse global outcomes obscure the causal signals needed for local policy refinement. We identify this bottleneck as a multi-agent credit assignment problem, which has long been studied in classical multi-agent reinforcement learning (MARL) but remains underaddressed in LLM-based systems. Building on this observation, we propose LangMARL, a framework that brings credit assignment and policy gradient evolution from cooperative MARL into the language space. LangMARL introduces agent-level language credit assignment, pioneers gradient evolution in language space for policy improvement, and summarizes task-relevant causal relations from replayed trajectories to provide dense feedback and improve convergence under sparse rewards. Extensive experiments across diverse cooperative multi-agent tasks demonstrate improved sample efficiency, interpretability, and strong generalization.

Related papers

• GlossoGen: Emergent Language in Complex Multi-Agent LLM Interactions [19.457187328571035]
  We introduce GlossoGen, a novel platform for studying multi-agent language evolution in complex scenarios.<n>We find that language evolution does occur between LLM agents, that the resulting languages are compositional and morphologically productive.<n>We observe that different conditions govern the transmission of language to new agents.
  arXiv  Detail & Related papers  (2026-09-01T16:22:07Z)
• Optimizing Language Models for Crosslingual Knowledge Consistency [90.86445137816942]
  Large language models are known to often exhibit inconsistent knowledge.<n>This is particularly problematic in multilingual scenarios, where models are likely to be asked similar questions in different languages.<n>In this work, we show that this issue can be mitigated using reinforcement learning with a structured reward function.
  arXiv  Detail & Related papers  (2026-03-04T23:36:55Z)
• Hierarchical Lead Critic based Multi-Agent Reinforcement Learning [1.4323566945483497]
  We introduce a novel sequential training scheme and MARL architecture, which learns from multiple perspectives on different hierarchy levels.<n>HLC demonstrates that introducing multiple hierarchies, leveraging local and global perspectives, can lead to improved performance with high sample efficiency and robust policies.
  arXiv  Detail & Related papers  (2026-02-25T08:33:39Z)
• MARTI-MARS$^2$: Scaling Multi-Agent Self-Search via Reinforcement Learning for Code Generation [64.2621682259008]
  Multi-Agent Reinforced Training and Inference Framework with Self-Search Scaling (MARTI-MARS2)<n>We propose a Multi-Agent Reinforced Training and Inference Framework with Self-Search Scaling (MARTI-MARS2) to integrate policy learning with multi-agent tree search.<n>We show that MARTI-MARS2 achieves 77.7%, outperforming strong baselines like GPT-5.1 on challenging code generation benchmarks.
  arXiv  Detail & Related papers  (2026-02-08T07:28:44Z)
• Natural Language Actor-Critic: Scalable Off-Policy Learning in Language Space [57.868527884634894]
  Natural Language Actor-Critic is a novel actor-critic algorithm that trains policies using natural language rather than scalar values.<n>We present results on a mixture of reasoning, web browsing, and tool-use with dialogue tasks, demonstrating that NLAC shows promise in outperforming existing training approaches.
  arXiv  Detail & Related papers  (2025-12-04T09:21:44Z)
• cMALC-D: Contextual Multi-Agent LLM-Guided Curriculum Learning with Diversity-Based Context Blending [3.939989712024268]
  Contextual MARL (cMARL) addresses this by parameterizing environments with context variables and training a context-agnostic policy.<n>Existing cMARL methods attempt to use curriculum learning to help train and evaluate context-agnostic policies.<n>We propose Contextual Multi-Agent LLM-Guided Curriculum Learning with Diversity-Based Context Blending (cMALC-D)
  arXiv  Detail & Related papers  (2025-08-28T14:16:17Z)
• Language-Driven Coordination and Learning in Multi-Agent Simulation Environments [5.139001420948807]
  This paper introduces a unified framework that incorporates large language models (LLMs) into multi-agent reinforcement learning (MARL)<n>The framework features three modular components of Coordinator, Communicator, and Memory, which dynamically generate subgoals.<n>It is evaluated in Google Research Football, MAgent Battle, and StarCraft II.
  arXiv  Detail & Related papers  (2025-06-01T06:46:49Z)
• LERO: LLM-driven Evolutionary framework with Hybrid Rewards and Enhanced Observation for Multi-Agent Reinforcement Learning [4.343021413805699]
  Multi-agent reinforcement learning (MARL) faces two critical bottlenecks distinct from single-agent RL.<n>We propose LERO, a framework integrating Large language models (LLMs) with evolutionary optimization to address these MARL-specific challenges.
  arXiv  Detail & Related papers  (2025-03-25T06:28:42Z)
• ReMA: Learning to Meta-think for LLMs with Multi-Agent Reinforcement Learning [53.817538122688944]
  We introduce Reinforced Meta-thinking Agents (ReMA) to elicit meta-thinking behaviors from Reasoning of Large Language Models (LLMs)<n>ReMA decouples the reasoning process into two hierarchical agents: a high-level meta-thinking agent responsible for generating strategic oversight and plans, and a low-level reasoning agent for detailed executions.<n> Empirical results from single-turn experiments demonstrate that ReMA outperforms single-agent RL baselines on complex reasoning tasks.
  arXiv  Detail & Related papers  (2025-03-12T16:05:31Z)
• Entropy-Regularized Token-Level Policy Optimization for Language Agent Reinforcement [67.1393112206885]
  Large Language Models (LLMs) have shown promise as intelligent agents in interactive decision-making tasks. We introduce Entropy-Regularized Token-level Policy Optimization (ETPO), an entropy-augmented RL method tailored for optimizing LLMs at the token level. We assess the effectiveness of ETPO within a simulated environment that models data science code generation as a series of multi-step interactive tasks.
  arXiv  Detail & Related papers  (2024-02-09T07:45:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.