Manifold-Constrained Sentence Embeddings via Triplet Loss: Projecting Semantics onto Spheres, Tori, and Möbius Strips

• URL: http://arxiv.org/abs/2505.00014v1
• Date: Tue, 22 Apr 2025 19:20:53 GMT
• Title: Manifold-Constrained Sentence Embeddings via Triplet Loss: Projecting Semantics onto Spheres, Tori, and Möbius Strips
• Authors: Vinit K. Chavan,
• Abstract summary: We introduce a novel framework that constrains sentence embeddings to lie on continuous manifold.<n>By enforcing differential geometric constraints on the output space, our approach encourages the learning of embeddings that are both discriminative and topologically structured.<n>Our results demonstrate that manifold-constrained embeddings, particularly those projected onto spheres and M"obius strips, significantly outperform traditional approaches in both clustering quality and classification performance.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in representation learning have emphasized the role of embedding geometry in capturing semantic structure. Traditional sentence embeddings typically reside in unconstrained Euclidean spaces, which may limit their ability to reflect complex relationships in language. In this work, we introduce a novel framework that constrains sentence embeddings to lie on continuous manifolds -- specifically the unit sphere, torus, and M\"obius strip -- using triplet loss as the core training objective. By enforcing differential geometric constraints on the output space, our approach encourages the learning of embeddings that are both discriminative and topologically structured. We evaluate our method on benchmark datasets (AG News and MBTI) and compare it to classical baselines including TF-IDF, Word2Vec, and unconstrained Keras-derived embeddings. Our results demonstrate that manifold-constrained embeddings, particularly those projected onto spheres and M\"obius strips, significantly outperform traditional approaches in both clustering quality (Silhouette Score) and classification performance (Accuracy). These findings highlight the value of embedding in manifold space -- where topological structure complements semantic separation -- offering a new and mathematically grounded direction for geometric representation learning in NLP.

Related papers

• TokenBlowUp: Resolving Representational Singularities in LLM Token Spaces via Monoidal Transformations [1.3824176915623292]
  Recent work has provided compelling evidence challenging the foundational manifold hypothesis for the token embedding spaces of Large Language Models.<n>We formalize this problem in the language of scheme theory and propose a rigorous resolution by applying the scheme-theoretic blow-up at each singular point.<n>We prove a formal theorem guaranteeing the geometric regularization of this new space, showing that the original pathologies are resolved.
  arXiv  Detail & Related papers  (2025-07-26T02:39:54Z)
• Unraveling the Localized Latents: Learning Stratified Manifold Structures in LLM Embedding Space with Sparse Mixture-of-Experts [3.9426000822656224]
  We conjecture that in large language models, the embeddings live in a local manifold structure with different dimensions depending on the perplexities and domains of the input data.<n>By incorporating an attention-based soft-gating network, we verify that our model learns specialized sub-manifolds for an ensemble of input data sources.
  arXiv  Detail & Related papers  (2025-02-19T09:33:16Z)
• Understanding and Mitigating Hyperbolic Dimensional Collapse in Graph Contrastive Learning [70.0681902472251]
  We propose a novel contrastive learning framework to learn high-quality graph embeddings in hyperbolic space.<n>Specifically, we design the alignment metric that effectively captures the hierarchical data-invariant information.<n>We show that in the hyperbolic space one has to address the leaf- and height-level uniformity related to properties of trees.
  arXiv  Detail & Related papers  (2023-10-27T15:31:42Z)
• Supervised Manifold Learning via Random Forest Geometry-Preserving Proximities [0.0]
  We show the weaknesses of class-conditional manifold learning methods quantitatively and visually. We propose an alternate choice of kernel for supervised dimensionality reduction using a data-geometry-preserving variant of random forest proximities.
  arXiv  Detail & Related papers  (2023-07-03T14:55:11Z)
• Linear Spaces of Meanings: Compositional Structures in Vision-Language Models [110.00434385712786]
  We investigate compositional structures in data embeddings from pre-trained vision-language models (VLMs) We first present a framework for understanding compositional structures from a geometric perspective. We then explain what these structures entail probabilistically in the case of VLM embeddings, providing intuitions for why they arise in practice.
  arXiv  Detail & Related papers  (2023-02-28T08:11:56Z)
• Variational Cross-Graph Reasoning and Adaptive Structured Semantics Learning for Compositional Temporal Grounding [143.5927158318524]
  Temporal grounding is the task of locating a specific segment from an untrimmed video according to a query sentence. We introduce a new Compositional Temporal Grounding task and construct two new dataset splits. We argue that the inherent structured semantics inside the videos and language is the crucial factor to achieve compositional generalization.
  arXiv  Detail & Related papers  (2023-01-22T08:02:23Z)
• Curved Geometric Networks for Visual Anomaly Recognition [39.91252195360767]
  Learning a latent embedding to understand the underlying nature of data distribution is often formulated in Euclidean spaces with zero curvature. In this work, we investigate benefits of the curved space for analyzing anomalies or out-of-distribution objects in data.
  arXiv  Detail & Related papers  (2022-08-02T01:15:39Z)
• Cross-Lingual BERT Contextual Embedding Space Mapping with Isotropic and Isometric Conditions [7.615096161060399]
  We investigate a context-aware and dictionary-free mapping approach by leveraging parallel corpora. Our findings unfold the tight relationship between isotropy, isometry, and isomorphism in normalized contextual embedding spaces.
  arXiv  Detail & Related papers  (2021-07-19T22:57:36Z)
• Deep Clustering by Semantic Contrastive Learning [67.28140787010447]
  We introduce a novel variant called Semantic Contrastive Learning (SCL) It explores the characteristics of both conventional contrastive learning and deep clustering. It can amplify the strengths of contrastive learning and deep clustering in a unified approach.
  arXiv  Detail & Related papers  (2021-03-03T20:20:48Z)
• Quadric hypersurface intersection for manifold learning in feature space [52.83976795260532]
  manifold learning technique suitable for moderately high dimension and large datasets. The technique is learned from the training data in the form of an intersection of quadric hypersurfaces. At test time, this manifold can be used to introduce an outlier score for arbitrary new points.
  arXiv  Detail & Related papers  (2021-02-11T18:52:08Z)
• Unsupervised Embedding of Hierarchical Structure in Euclidean Space [30.507049058838025]
  We consider learning a non-linear embedding of data into Euclidean space as a way to improve the hierarchical clustering produced by agglomerative algorithms. We show that rescaling the latent space embedding leads to improved results for both dendrogram purity and the Moseley-Wang cost function.
  arXiv  Detail & Related papers  (2020-10-30T03:57:09Z)
• A Comparative Study on Structural and Semantic Properties of Sentence Embeddings [77.34726150561087]
  We propose a set of experiments using a widely-used large-scale data set for relation extraction. We show that different embedding spaces have different degrees of strength for the structural and semantic properties. These results provide useful information for developing embedding-based relation extraction methods.
  arXiv  Detail & Related papers  (2020-09-23T15:45:32Z)
• APo-VAE: Text Generation in Hyperbolic Space [116.11974607497986]
  In this paper, we investigate text generation in a hyperbolic latent space to learn continuous hierarchical representations. An Adrial Poincare Variversaational Autoencoder (APo-VAE) is presented, where both the prior and variational posterior of latent variables are defined over a Poincare ball via wrapped normal distributions. Experiments in language modeling and dialog-response generation tasks demonstrate the winning effectiveness of the proposed APo-VAE model.
  arXiv  Detail & Related papers  (2020-04-30T19:05:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.