Token-to-Token Alignment of Text Embeddings for Semantic Blending

• URL: http://arxiv.org/abs/2606.24021v1
• Date: Mon, 22 Jun 2026 23:54:40 GMT
• Title: Token-to-Token Alignment of Text Embeddings for Semantic Blending
• Abstract summary: We introduce Token-to-Token alignment, a framework that establishes explicit semantic correspondence between tokens across prompts.<n>Our results show that text embedding spaces in text-to-image models implicitly encode a continuous semantic structure that becomes accessible once representations are properly aligned.
• Score: 55.88554156801394
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In modern generative models, images are specified and controlled through text prompts. In practice, images are generated from sequences of tokens derived from these prompts. However, the space of token sequences lacks a consistent accessible structure: semantically similar images may correspond to sequences that differ in wording, ordering, and placement of concepts, while similar token sequences may encode very different semantics. This apparent lack of structure makes it difficult to perform smooth transitions in this space, hindering applications such as image blending and continuous control of edits. We argue that this limitation stems not from the absence of semantic structure, but from misalignment between representations. To address this misalignment, we introduce Token-to-Token alignment, a framework that establishes explicit semantic correspondence between tokens across prompts. Our approach transforms prompts into a structured representation in which semantically corresponding concepts are mapped to consistent positions across prompts, and then aligns their token embeddings based on semantic similarity. Concretely, the method consists of two stages: a structural alignment that rephrases prompts into a shared structured form, followed by an embedding-level alignment that matches token representations across prompts. With this alignment in place, simple linear interpolation becomes a meaningful operation, producing smooth and coherent semantic transitions and enabling applications such as blending and continuous editing. Our results show that text embedding spaces in text-to-image models implicitly encode a continuous semantic structure that becomes accessible once representations are properly aligned, suggesting that semantic control can be achieved by organizing existing representations rather than modifying the generative model.

Related papers

• STEDiff: Strengthening Text Embedding for Text-to-Image Alignment in Diffusion Model [4.685207648946472]
  STEDiff is a training-free method designed to enhance semantic representations directly within the text-embedding space.<n>Our method notably improves semantic consistency and generation in complex scenarios.
  arXiv  Detail & Related papers  (2026-06-09T09:59:09Z)
• MaskAlign: Token-Subset Representation Alignment for Efficient Diffusion Training [53.777757218665386]
  MaskAlign is a token-subset representation alignment method that applies alignment to randomly sampled token subsets during training.<n>We show that MaskAlign reduces the dependence of representation alignment on the complete token set and encourages alignment behavior that is more stable under token-subset perturbations.
  arXiv  Detail & Related papers  (2026-06-07T19:11:22Z)
• Decomposed Vision-Language Alignment for Fine-Grained Open-Vocabulary Segmentation [16.594496639787195]
  Open-vocabulary segmentation models often struggle to generalize to unseen combinations of categories object and attributes.<n>We propose a Decomposed Vision-Language Alignment framework that factorizes textual prompts into a concept token and multiple attribute tokens.<n>At the feature level, we introduce a Feature-Gated Cross-Attention module that generates attribute-specific gating maps to fuse information in a multiplicative manner.
  arXiv  Detail & Related papers  (2026-05-15T13:27:01Z)
• Communication-Inspired Tokenization for Structured Image Representations [74.17163003465537]
  COMmunication inspired Tokenization (COMiT) is a framework for learning structured discrete visual token sequences.<n>Our experiments demonstrate that while semantic alignment provides grounding, attentive sequential tokenization is critical for inducing interpretable, object-centric token structure.
  arXiv  Detail & Related papers  (2026-02-24T09:53:50Z)
• TokenCLIP: Token-wise Prompt Learning for Zero-shot Anomaly Detection [62.95726973851089]
  TokenCLIP is a token-wise adaptation framework for anomaly learning.<n>It enables dynamic alignment between visual and learnable textual spaces for fine-grained anomaly learning.
  arXiv  Detail & Related papers  (2025-10-24T05:51:31Z)
• InvSeg: Test-Time Prompt Inversion for Semantic Segmentation [33.60580908728705]
  InvSeg is a test-time prompt inversion method that tackles open-vocabulary semantic segmentation.<n>We introduce Contrastive Soft Clustering (CSC) to align derived masks with the image's structure information.<n>InvSeg learns context-rich text prompts in embedding space and achieves accurate semantic alignment across modalities.
  arXiv  Detail & Related papers  (2024-10-15T10:20:31Z)
• RealignDiff: Boosting Text-to-Image Diffusion Model with Coarse-to-fine Semantic Re-alignment [112.45442468794658]
  We propose a two-stage coarse-to-fine semantic re-alignment method, named RealignDiff. In the coarse semantic re-alignment phase, a novel caption reward is proposed to evaluate the semantic discrepancy between the generated image caption and the given text prompt. The fine semantic re-alignment stage employs a local dense caption generation module and a re-weighting attention modulation module to refine the previously generated images from a local semantic view.
  arXiv  Detail & Related papers  (2023-05-31T06:59:21Z)
• Unsupervised Distillation of Syntactic Information from Contextualized Word Representations [62.230491683411536]
  We tackle the task of unsupervised disentanglement between semantics and structure in neural language representations. To this end, we automatically generate groups of sentences which are structurally similar but semantically different. We demonstrate that our transformation clusters vectors in space by structural properties, rather than by lexical semantics.
  arXiv  Detail & Related papers  (2020-10-11T15:13:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.