Direct Semantic Communication Between Large Language Models via Vector Translation

• URL: http://arxiv.org/abs/2511.03945v1
• Date: Thu, 06 Nov 2025 00:43:29 GMT
• Title: Direct Semantic Communication Between Large Language Models via Vector Translation
• Abstract summary: Large language models (LLMs) pass messages as plain tokens, discarding most latent semantics.<n>We form a latent bridge via vector translations, which use learned mappings that enable direct semantic exchange.
• Score: 3.81908263930559
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In multi-agent settings, such as debate, reflection, or tool-calling, large language models (LLMs) pass messages as plain tokens, discarding most latent semantics. This constrains information transfer and adds unnecessary computational overhead. We form a latent bridge via vector translations, which use learned mappings that enable direct semantic exchange between representation spaces. A dual-encoder translator trained between Llama-2-7B and Mistral-7B-Instruct attains an average cosine alignment of 0.538. Injecting the translated vectors at 30 percent blending strength steers the target model's generation without destabilizing logits. Bidirectional evaluation shows a 2.01:1 transfer asymmetry, indicating that general-purpose models yield more transferable representations than instruction-tuned variants. This conservative injection preserves computational stability while demonstrating that cross-model latent communication is feasible, enabling collaborative AI systems that share meaning rather than tokens.

Related papers

• Architecture-Dependent Causal Transfer of Activation States Across Large Language Models [51.56484100374058]
  Internal language activation states can be transferred causally between different large language models.<n>Injecting projected activations into a target model during generation produces a statistically significant, preregistered causal effect.<n>We conclude that end-to-end activation-state transfer between LLMs, as currently implemented, is architecture-dependent rather than universal.
  arXiv  Detail & Related papers  (2026-08-17T09:53:27Z)
• StateBridge: Training-free Hidden-state Alignment for Latent Communication in LLM Multi-Agent Systems [35.74603358195608]
  We propose a training-free latent communication approach that aligns the sender's hidden states to the receiver's input space via a closed-form transformation.<n>StateBridge achieves the best or tied-best score on 22 out of 26 model-task pairs.
  arXiv  Detail & Related papers  (2026-08-13T14:40:59Z)
• Do Unified Multimodal Models Think in One Space? A Lens Through Cross-Branch Steering [7.62772056485722]
  We introduce emphcross-branch semantic steering, an intervention-based framework that extracts semantic directions from one branch and applies them to the other.<n>We show that steering vectors learned from the understanding branch can transfer to generation, enabling controllable image synthesis and improved semantic faithfulness.<n>Our results reveal that architectural unification does not guarantee semantic alignment, and establish cross-branch steering as a practical tool for probing multimodal representations.
  arXiv  Detail & Related papers  (2026-07-29T02:46:20Z)
• Bilingual Text-to-Motion Generation: A New Benchmark and Baselines [52.71312720094036]
  We introduce BiHumanML3D, the first bilingual text-to-motion benchmark, constructed via LLM-assisted annotation and rigorous manual correction.<n>We also propose a simple yet effective baseline, Bilingual Motion Diffusion synthesis (BiMD), featuring Cross-Lingual Alignment (CLA)<n>BiMD with CLA achieves an FID of 0.045 vs. 0.169 and R@3 of 82.8% vs. 80.8%, significantly outperforms monolingual diffusion models and translation baselines on BiHumanML3D.
  arXiv  Detail & Related papers  (2026-03-26T08:48:27Z)
• Interference Matrix: Quantifying Cross-Lingual Interference in Transformer Encoders [55.749883010057545]
  We construct an interference matrix by training and evaluating small BERT-like models on all possible language pairs.<n>Our analysis reveals that interference between languages is asymmetrical and that its patterns do not align with traditional linguistic characteristics.
  arXiv  Detail & Related papers  (2025-08-04T10:02:19Z)
• Augmenting Multi-Agent Communication with State Delta Trajectory [44.68954502805962]
  We propose a new communication protocol that transfers both natural language tokens and token-wise state transition from one agent to another.<n>We find that the sequence of state changes in LLMs after generating each token can better reflect the information hidden behind the inference process.<n>The experimental results show that multi-agent systems with SDE achieve SOTA performance compared to other communication protocols.
  arXiv  Detail & Related papers  (2025-06-24T00:38:25Z)
• Relative Representations of Latent Spaces enable Efficient Semantic Channel Equalization [11.052047963214006]
  We present a novel semantic equalization algorithm that enables communication between agents with different languages without additional retraining.<n>Our numerical results show the effectiveness of the proposed approach allowing seamless communication between agents with radically different models.
  arXiv  Detail & Related papers  (2024-11-29T14:08:48Z)
• Language Driven Occupancy Prediction [13.35971455725581]
  We introduce LOcc, an effective and generalizable framework for open-vocabulary occupancy prediction.<n>Our pipeline presents a feasible way to dig into the valuable semantic information of images, transferring text labels from images to LiDAR point clouds and ultimately to voxels.<n>By replacing the original prediction head of supervised occupancy models with a geometry head for binary occupancy states and a language head for language features, LOcc effectively uses the generated language ground truth to guide the learning of 3D language volume.
  arXiv  Detail & Related papers  (2024-11-25T03:47:10Z)
• VECO 2.0: Cross-lingual Language Model Pre-training with Multi-granularity Contrastive Learning [56.47303426167584]
  We propose a cross-lingual pre-trained model VECO2.0 based on contrastive learning with multi-granularity alignments. Specifically, the sequence-to-sequence alignment is induced to maximize the similarity of the parallel pairs and minimize the non-parallel pairs. token-to-token alignment is integrated to bridge the gap between synonymous tokens excavated via the thesaurus dictionary from the other unpaired tokens in a bilingual instance.
  arXiv  Detail & Related papers  (2023-04-17T12:23:41Z)
• Improving Zero-Shot Multilingual Translation with Universal Representations and Cross-Mappings [23.910477693942905]
  Improved zero-shot translation requires the model to learn universal representations and cross-mapping relationships. We propose the state's distance based on the optimal theory to model the difference of the representations output by the encoder. We propose an agreement-based training scheme, which can help the model make consistent predictions.
  arXiv  Detail & Related papers  (2022-10-28T02:47:05Z)
• Exploring Unsupervised Pretraining Objectives for Machine Translation [99.5441395624651]
  Unsupervised cross-lingual pretraining has achieved strong results in neural machine translation (NMT) Most approaches adapt masked-language modeling (MLM) to sequence-to-sequence architectures, by masking parts of the input and reconstructing them in the decoder. We compare masking with alternative objectives that produce inputs resembling real (full) sentences, by reordering and replacing words based on their context.
  arXiv  Detail & Related papers  (2021-06-10T10:18:23Z)
• Tight Integrated End-to-End Training for Cascaded Speech Translation [40.76367623739673]
  A cascaded speech translation model relies on discrete and non-differentiable transcription. Direct speech translation is an alternative method to avoid error propagation. This work explores the feasibility of collapsing the entire cascade components into a single end-to-end trainable model.
  arXiv  Detail & Related papers  (2020-11-24T15:43:49Z)
• Learning Source Phrase Representations for Neural Machine Translation [65.94387047871648]
  We propose an attentive phrase representation generation mechanism which is able to generate phrase representations from corresponding token representations. In our experiments, we obtain significant improvements on the WMT 14 English-German and English-French tasks on top of the strong Transformer baseline.
  arXiv  Detail & Related papers  (2020-06-25T13:43:11Z)
• From Zero to Hero: On the Limitations of Zero-Shot Cross-Lingual Transfer with Multilingual Transformers [62.637055980148816]
  Massively multilingual transformers pretrained with language modeling objectives have become a de facto default transfer paradigm for NLP. We show that cross-lingual transfer via massively multilingual transformers is substantially less effective in resource-lean scenarios and for distant languages.
  arXiv  Detail & Related papers  (2020-05-01T22:04:58Z)
• Improve Variational Autoencoder for Text Generationwith Discrete Latent Bottleneck [52.08901549360262]
  Variational autoencoders (VAEs) are essential tools in end-to-end representation learning. VAEs tend to ignore latent variables with a strong auto-regressive decoder. We propose a principled approach to enforce an implicit latent feature matching in a more compact latent space.
  arXiv  Detail & Related papers  (2020-04-22T14:41:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.