Beyond tokens: a unified framework for latent communication in LLM-based multi-agent systems

• URL: http://arxiv.org/abs/2606.05711v2
• Date: Fri, 05 Jun 2026 14:53:01 GMT
• Title: Beyond tokens: a unified framework for latent communication in LLM-based multi-agent systems
• Abstract summary: Multi-agent systems built on large language models (LLMs) have become a prevailing paradigm for tackling complex reasoning, planning, and tool-use tasks.<n>The dominant communication protocol in such systems is natural language: agents exchange messages token-by-token, verbalising their internal reasoning so that peers can read, verify, and respond.<n>A growing body of work therefore explores an alternative protocol -- latent communication, in which agents exchange continuous representations directly, bypassing the bottleneck of text generation.<n>This paper presents a unified framework for organising the rapidly expanding literature on latent communication.
• Score: 0.65268245109828
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multi-agent systems built on large language models (LLMs) have become a prevailing paradigm for tackling complex reasoning, planning, and tool-use tasks. The dominant communication protocol in such systems is natural language: agents exchange messages token-by-token, verbalising their internal reasoning so that peers can read, verify, and respond. While convenient and interpretable, this protocol suffers from three structural drawbacks -- high inference cost, irreversible information loss during discretization, and ambiguity/redundancy of natural language. A growing body of work therefore explores an alternative protocol -- latent communication -- in which agents exchange continuous representations (embeddings, hidden states, or KV-caches) directly, bypassing the bottleneck of text generation. This paper presents a unified framework for organising the rapidly expanding literature on latent communication. We analyse existing methods along three orthogonal axes: (1) WHAT information is communicated (Embeddings, Hidden States, KV-Caches, or other continuous state); (2) WHICH sender-receiver alignment is used (latent-space alignment and layer alignment); and (3) HOW the communicated information is fused into the receiver (concatenation, prepending, mathematical operations, cross-attention, or cache restoration). Under this 3-axis framework, we systematically categorise eighteen representative methods proposed between 2024 and 2026, identify five major design patterns, and surface a set of open challenges -- including cross-architecture alignment, security of latent channels, compression for edge deployment, and the relationship between latent communication and latent chain-of-thought. We hope that this framework both lowers the barrier to entry for new researchers and provides a vocabulary for comparing future work.

Related papers

• From Semantic to Token Communication: The Next Paradigm for Large-Model-Driven 6G Intelligent Connectivity [94.51565447227847]
  6G networks are driving communication systems from reliable bit delivery toward meaning-aware and task-oriented connectivity.<n>Large models (LMs) with strong multimodal understanding and generation capabilities have accelerated this shift.<n>Current LM-driven semantic communication (SemCom) remains fragmented.<n>We argue that tokens provide a natural candidate for this missing abstraction.
  arXiv  Detail & Related papers  (2026-09-09T18:11:38Z)
• A Technical Taxonomy of LLM Agent Communication Protocols [60.76747983053368]
  This study develops a technical taxonomy to classify and analyze large language models (LLMs) agent communication protocols.<n>The framework guides protocol selection and highlights open research gaps such as privacy and policy enforcement.
  arXiv  Detail & Related papers  (2026-06-17T14:45:20Z)
• HyLaT: Efficient Multi-Agent Communication via Hybrid Latent-Text Protocol [54.411053382333996]
  HyLaT is a hybrid latent-text communication protocol that transmits elaborate cognitive signals through a latent channel for efficiency.<n>We introduce a two-stage training framework combining single-agent hybrid generation learning and multi-agent interactive co-training, enabling agents to generate and interpret hybrid messages.<n>Experiments demonstrate that HyLaT reduces communication overhead significantly while maintaining competitive task performance, with strong generalization and robustness across diverse settings.
  arXiv  Detail & Related papers  (2026-05-25T04:50:15Z)
• Beyond Message Passing: A Semantic View of Agent Communication Protocols [19.409127393922216]
  Agent communication protocols are becoming critical infrastructure for large language model (LLM) systems.<n>This work presents a human-inspired perspective on this emerging landscape by organizing agent communication into three layers: communication, syntactic, and semantic.
  arXiv  Detail & Related papers  (2026-03-30T00:40:30Z)
• A Compressive-Expressive Communication Framework for Compositional Representations [0.6937243101289334]
  We introduce CELEBI, a self-supervised framework for inducing compositionality in learned representations from pre-trained models.<n>Our method significantly improves both the efficiency and compositionality of the learned messages on the Shapes3D and MPI3D datasets.<n>This work provides new theoretical and empirical evidence for the emergence of structured, generalizable communication protocols from simplicity-based inductive biases.
  arXiv  Detail & Related papers  (2025-01-31T14:46:11Z)
• Layered Chain-of-Thought Prompting for Multi-Agent LLM Systems: A Comprehensive Approach to Explainable Large Language Models [0.0]
  We propose Layered Chain-of-Thought (Layered-CoT) Prompting, a novel framework that systematically segments the reasoning process into multiple layers.<n>We present three scenarios -- medical triage, financial risk assessment, and agile engineering -- and demonstrate how Layered-CoT surpasses vanilla CoT in terms of transparency, correctness, and user engagement.
  arXiv  Detail & Related papers  (2025-01-29T13:21:09Z)
• Bidirectional Emergent Language in Situated Environments [4.950411915351642]
  We introduce two novel cooperative environments: Multi-Agent Pong and Collectors. optimal performance requires the emergence of a communication protocol, but moderate success can be achieved without one. We find that the emerging communication is sparse, with the agents only generating meaningful messages and acting upon incoming messages in states where they cannot succeed without coordination.
  arXiv  Detail & Related papers  (2024-08-26T21:25:44Z)
• Agent-driven Generative Semantic Communication with Cross-Modality and Prediction [57.335922373309074]
  We propose a novel agent-driven generative semantic communication framework based on reinforcement learning. In this work, we develop an agent-assisted semantic encoder with cross-modality capability, which can track the semantic changes, channel condition, to perform adaptive semantic extraction and sampling. The effectiveness of the designed models has been verified using the UA-DETRAC dataset, demonstrating the performance gains of the overall A-GSC framework.
  arXiv  Detail & Related papers  (2024-04-10T13:24:27Z)
• Language-Oriented Communication with Semantic Coding and Knowledge Distillation for Text-to-Image Generation [53.97155730116369]
  We put forward a novel framework of language-oriented semantic communication (LSC) In LSC, machines communicate using human language messages that can be interpreted and manipulated via natural language processing (NLP) techniques for SC efficiency. We introduce three innovative algorithms: 1) semantic source coding (SSC), which compresses a text prompt into its key head words capturing the prompt's syntactic essence; 2) semantic channel coding ( SCC), that improves robustness against errors by substituting head words with their lenghthier synonyms; and 3) semantic knowledge distillation (SKD), that produces listener-customized prompts via in-context learning the listener's
  arXiv  Detail & Related papers  (2023-09-20T08:19:05Z)
• Cognitive Semantic Communication Systems Driven by Knowledge Graph: Principle, Implementation, and Performance Evaluation [74.38561925376996]
  Two cognitive semantic communication frameworks are proposed for the single-user and multiple-user communication scenarios. An effective semantic correction algorithm is proposed by mining the inference rule from the knowledge graph. For the multi-user cognitive semantic communication system, a message recovery algorithm is proposed to distinguish messages of different users.
  arXiv  Detail & Related papers  (2023-03-15T12:01:43Z)
• Learning Structured Communication for Multi-agent Reinforcement Learning [104.64584573546524]
  This work explores the large-scale multi-agent communication mechanism under a multi-agent reinforcement learning (MARL) setting. We propose a novel framework termed as Learning Structured Communication (LSC) by using a more flexible and efficient communication topology.
  arXiv  Detail & Related papers  (2020-02-11T07:19:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.