Learning Evolving Latent Strategies for Multi-Agent Language Systems without Model Fine-Tuning

• URL: http://arxiv.org/abs/2512.20629v1
• Date: Fri, 28 Nov 2025 23:36:45 GMT
• Title: Learning Evolving Latent Strategies for Multi-Agent Language Systems without Model Fine-Tuning
• Abstract summary: This study proposes a multi-agent language framework that enables continual strategy evolution without fine-tuning the language model's parameters.<n>We construct a dual-loop architecture: the behavior loop adjusts action preferences based on environmental rewards, while the language loop updates the external latent vectors by reflecting on the semantic embeddings of generated text.
• Score: 0.6345523830122167
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This study proposes a multi-agent language framework that enables continual strategy evolution without fine-tuning the language model's parameters. The core idea is to liberate the latent vectors of abstract concepts from traditional static semantic representations, allowing them to be continuously updated through environmental interaction and reinforcement feedback. We construct a dual-loop architecture: the behavior loop adjusts action preferences based on environmental rewards, while the language loop updates the external latent vectors by reflecting on the semantic embeddings of generated text. Together, these mechanisms allow agents to develop stable and disentangled strategic styles over long-horizon multi-round interactions. Experiments show that agents' latent spaces exhibit clear convergence trajectories under reflection-driven updates, along with structured shifts at critical moments. Moreover, the system demonstrates an emergent ability to implicitly infer and continually adapt to emotional agents, even without shared rewards. These results indicate that, without modifying model parameters, an external latent space can provide language agents with a low-cost, scalable, and interpretable form of abstract strategic representation.

Related papers

• Measure Twice, Click Once: Co-evolving Proposer and Visual Critic via Reinforcement Learning for GUI Grounding [44.242287418575664]
  Graphical User Interface (GUI) grounding requires mapping natural language instructions to precise pixel coordinates.<n>We propose replacing static consistency strategies with a learnable selection mechanism that selects the optimal target by critiquing its own proposals rendered on the screenshot.
  arXiv  Detail & Related papers  (2026-04-23T04:23:31Z)
• Multimodal Latent Reasoning via Hierarchical Visual Cues Injection [16.779425236020433]
  This work posits that robust reasoning should evolve within a latent space, integrating multimodal signals seamlessly.<n>We propose a novel framework that instills deliberate, "slow thinking" without depending on superficial textual rationales.<n>We show that test-time scaling is effective when incorporating vision knowledge, and that integrating hierarchical information significantly enhances the model's understanding of complex scenes.
  arXiv  Detail & Related papers  (2026-02-05T06:31:12Z)
• Top 10 Open Challenges Steering the Future of Diffusion Language Model and Its Variants [85.33837131101342]
  We propose a strategic roadmap organized into four pillars: foundational infrastructure, algorithmic optimization, cognitive reasoning, and unified multimodal intelligence.<n>We argue that this transition is essential for developing next-generation AI capable of complex structural reasoning, dynamic self-correction, and seamless multimodal integration.
  arXiv  Detail & Related papers  (2026-01-20T14:58:23Z)
• Beyond Pipelines: A Survey of the Paradigm Shift toward Model-Native Agentic AI [27.209787026732972]
  The rapid evolution of agentic AI marks a new phase in artificial intelligence.<n>This survey traces the paradigm shift in building agentic AI.<n>It examines how each capability has evolved from externally scripted modules to end-to-end learned behaviors.
  arXiv  Detail & Related papers  (2025-10-19T05:23:43Z)
• Sample-Efficient Online Learning in LM Agents via Hindsight Trajectory Rewriting [92.57796055887995]
  We introduce ECHO, a prompting framework that adapts hindsight experience replay from reinforcement learning for language model agents.<n> ECHO generates optimized trajectories for alternative goals that could have been achieved during failed attempts.<n>We evaluate ECHO on stateful versions of XMiniGrid, a text-based navigation and planning benchmark, and PeopleJoinQA, a collaborative information-gathering enterprise simulation.
  arXiv  Detail & Related papers  (2025-10-11T18:11:09Z)
• Generative World Models of Tasks: LLM-Driven Hierarchical Scaffolding for Embodied Agents [0.0]
  We propose an effective world model for decision-making that models the world's physics and its task semantics.<n>A systematic review of 2024 research in low-resource multi-agent soccer reveals a clear trend towards integrating symbolic and hierarchical methods.<n>We formalize this trend into a framework for Hierarchical Task Environments (HTEs), which are essential for bridging the gap between simple, reactive behaviors and sophisticated, strategic team play.
  arXiv  Detail & Related papers  (2025-09-05T01:03:51Z)
• The Landscape of Agentic Reinforcement Learning for LLMs: A Survey [103.32591749156416]
  The emergence of agentic reinforcement learning (Agentic RL) marks a paradigm shift from conventional reinforcement learning applied to large language models (LLM RL)<n>This survey formalizes this conceptual shift by contrasting the degenerate single-step Markov Decision Processes (MDPs) of LLM-RL with the temporally extended, partially observable Markov decision processes (POMDPs) that define Agentic RL.
  arXiv  Detail & Related papers  (2025-09-02T17:46:26Z)
• Flexible Language Modeling in Continuous Space with Transformer-based Autoregressive Flows [46.673228292287895]
  We propose a novel framework that employs transformer-based autoregressive normalizing flows to model continuous representations.<n>This approach unlocks substantial flexibility, enabling the construction of models that can capture global bi-directional context.<n>We propose new mixture-based coupling transformations designed to capture complex dependencies within the latent space shaped by discrete data.
  arXiv  Detail & Related papers  (2025-07-01T04:51:25Z)
• LangSuitE: Planning, Controlling and Interacting with Large Language Models in Embodied Text Environments [70.91258869156353]
  We introduce LangSuitE, a versatile and simulation-free testbed featuring 6 representative embodied tasks in textual embodied worlds. Compared with previous LLM-based testbeds, LangSuitE offers adaptability to diverse environments without multiple simulation engines. We devise a novel chain-of-thought (CoT) schema, EmMem, which summarizes embodied states w.r.t. history information.
  arXiv  Detail & Related papers  (2024-06-24T03:36:29Z)
• Retroformer: Retrospective Large Language Agents with Policy Gradient Optimization [103.70896967077294]
  This paper introduces a principled framework for reinforcing large language agents by learning a retrospective model. Our proposed agent architecture learns from rewards across multiple environments and tasks, for fine-tuning a pre-trained language model. Experimental results on various tasks demonstrate that the language agents improve over time.
  arXiv  Detail & Related papers  (2023-08-04T06:14:23Z)
• Shattering the Agent-Environment Interface for Fine-Tuning Inclusive Language Models [24.107358120517336]
  In this work, we adopt a novel perspective wherein a pre-trained language model is itself simultaneously a policy, reward function, and transition function. An immediate consequence of this is that reward learning and language model fine-tuning can be performed jointly and directly, without requiring any further downstream policy optimization.
  arXiv  Detail & Related papers  (2023-05-19T06:21:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.