How Far Do Simple Transformations Translate Across Text Embedding Models?

• URL: http://arxiv.org/abs/2608.05980v1
• Date: Thu, 06 Aug 2026 12:57:01 GMT
• Title: How Far Do Simple Transformations Translate Across Text Embedding Models?
• Abstract summary: We investigate whether simple transformations can translate representations across heterogeneous text embedding models.<n>We test the hypothesis of latent universality in a realistic text setting beyond simplified benchmarks.<n>Overall, the results show that heterogeneous embedding spaces are not universally related by simple mappings as often suggested in some literature.
• Score: 1.7812673697676977
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We investigate whether simple transformations can translate representations across heterogeneous text embedding models. Understanding how independently trained models organize semantic information is an enabler for AI-to-AI latent communication without decoding into human-readable text. Focusing on lightweight translators such as linear mappings, we test the literature hypothesis of latent universality in a realistic text setting beyond simplified benchmarks. Across nine embedding models differing in architecture, pooling strategy, and training objective, we evaluate compatibility using CKA, downstream transfer, fidelity, and retrieval. Simple translators recover meaningful shared structure and support transfer for some compatible pairs, but fail sharply for others. Compatibility depends jointly on architecture, training objective, pooling, and data distribution. Overall, the results show that heterogeneous embedding spaces are not universally related by simple mappings as often suggested in some literature.

Related papers

• Does Linguistic Structure Enrichment Enhance Coherence Assessment? Not With Current Architectures [1.756239703835073]
  We investigate whether enriching text with syntactic and rhetorical information can improve incoherence prediction.<n>Experiments and analysis show that plain texts achieved higher accuracy because the added information was structurally and syntactically incompatible with the language model's architecture.
  arXiv  Detail & Related papers  (2026-09-09T22:53:50Z)
• TextLDM: Language Modeling with Continuous Latent Diffusion [89.69255520673248]
  Diffusion Transformers (DiT) trained with flow matching in a VAE latent space have unified visual generation across images and videos.<n>We propose TextLDM, which transfers the visual latent diffusion recipe to text generation with minimal architectural modification.
  arXiv  Detail & Related papers  (2026-05-08T13:54:34Z)
• Zero-Shot Synthetic-to-Real Handwritten Text Recognition via Task Analogies [60.84720484975815]
  Handwritten Text Recognition (HTR) models often struggle to generalize to real text.<n>We tackle the fully zero-shot synthetic-to-real generalization setting.<n>Our approach learns how model parameters change when moving from synthetic to real handwriting in one or more source languages.
  arXiv  Detail & Related papers  (2026-04-08T07:33:31Z)
• Explicit Grammar Semantic Feature Fusion for Robust Text Classification [0.0]
  Natural Language Processing enables computers to understand human language by analysing and classifying text efficiently.<n>Existing models capture features by learning from large corpora with transformer models, which are computationally intensive and unsuitable for resource-constrained environments.<n>Our proposed study incorporates comprehensive grammatical rules alongside semantic information to build a robust, lightweight classification model.
  arXiv  Detail & Related papers  (2026-02-24T10:25:29Z)
• Negative Matters: Multi-Granularity Hard-Negative Synthesis and Anchor-Token-Aware Pooling for Enhanced Text Embeddings [25.565372681837697]
  We introduce a Multi-Granularity Hard-negative (MGH) synthesis framework to generate diverse negative samples with varying levels of similarity with the query.<n>We also propose an Anchor Token Aware (ATA) pooling method to improve text embedding accuracy without increasing model complexity.
  arXiv  Detail & Related papers  (2025-08-31T13:24:48Z)
• Contextually Guided Transformers via Low-Rank Adaptation [14.702057924366345]
  Large Language Models (LLMs) based on Transformers excel at text processing, but their reliance on prompts for specialized behavior introduces computational overhead.<n>We propose a modification to a Transformer architecture that eliminates the need for explicit prompts by learning to encode context into the model's weights.
  arXiv  Detail & Related papers  (2025-06-06T01:34:39Z)
• Mind the Gap: A Generalized Approach for Cross-Modal Embedding Alignment [0.0]
  Retrieval-Augmented Generation (RAG) systems retrieve context across different text modalities due to semantic gaps. We introduce a generalized projection-based method, inspired by adapter modules in transfer learning, that efficiently bridges these gaps. Our approach emphasizes speed, accuracy, and data efficiency, requiring minimal resources for training and inference.
  arXiv  Detail & Related papers  (2024-10-30T20:28:10Z)
• Text-Video Retrieval with Global-Local Semantic Consistent Learning [122.15339128463715]
  We propose a simple yet effective method, Global-Local Semantic Consistent Learning (GLSCL) GLSCL capitalizes on latent shared semantics across modalities for text-video retrieval. Our method achieves comparable performance with SOTA as well as being nearly 220 times faster in terms of computational cost.
  arXiv  Detail & Related papers  (2024-05-21T11:59:36Z)
• Explaining Text Similarity in Transformer Models [52.571158418102584]
  Recent advances in explainable AI have made it possible to mitigate limitations by leveraging improved explanations for Transformers. We use BiLRP, an extension developed for computing second-order explanations in bilinear similarity models, to investigate which feature interactions drive similarity in NLP models. Our findings contribute to a deeper understanding of different semantic similarity tasks and models, highlighting how novel explainable AI methods enable in-depth analyses and corpus-level insights.
  arXiv  Detail & Related papers  (2024-05-10T17:11:31Z)
• How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVEN [63.79300884115027]
  Current language models can generate high-quality text. Are they simply copying text they have seen before, or have they learned generalizable linguistic abstractions? We introduce RAVEN, a suite of analyses for assessing the novelty of generated text.
  arXiv  Detail & Related papers  (2021-11-18T04:07:09Z)
• Syntax-Enhanced Pre-trained Model [49.1659635460369]
  We study the problem of leveraging the syntactic structure of text to enhance pre-trained models such as BERT and RoBERTa. Existing methods utilize syntax of text either in the pre-training stage or in the fine-tuning stage, so that they suffer from discrepancy between the two stages. We present a model that utilizes the syntax of text in both pre-training and fine-tuning stages.
  arXiv  Detail & Related papers  (2020-12-28T06:48:04Z)
• Two-Level Transformer and Auxiliary Coherence Modeling for Improved Text Segmentation [9.416757363901295]
  We introduce a novel supervised model for text segmentation with simple but explicit coherence modeling. Our model -- a neural architecture consisting of two hierarchically connected Transformer networks -- is a multi-task learning model that couples the sentence-level segmentation objective with the coherence objective that differentiates correct sequences of sentences from corrupt ones.
  arXiv  Detail & Related papers  (2020-01-03T17:06:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.