Geometry of Semantic Space: Comparative Study of Discrete and Continuous Models

• URL: http://arxiv.org/abs/2606.07183v1
• Date: Fri, 05 Jun 2026 11:47:22 GMT
• Title: Geometry of Semantic Space: Comparative Study of Discrete and Continuous Models
• Abstract summary: We compare supervised vector embeddings with lexical co-occurrence graphs that encode semantic relations more directly.<n>In contrast, graph-based models reveal a clearer and more human-readable organization of meaning.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This work examines the semantic geometry underlying NLP models. We compare supervised vector embeddings, such as CamemBERT, with lexical co-occurrence graphs that encode semantic relations more directly. While transformer-based embeddings achieve strong performance, their induced geometries often display unsatisfactory distributions. In contrast, graph-based models reveal a clearer and more human-readable organization of meaning. We have implemented a methodology that allows us to perform a comparative analysis either based on the structure of the graphs or based on the topology of the embeddings induced by these two approaches. The results of the comparison -- applied to the French "Great National Debate" corpus a collection of citizen contributions to the public debate -- show a similar local topology but a very different overall structure and topology. Theses findings suggest complementary perspectives between deep supervised models and graph-based models, considering a new pathway to guide neural architectures toward more stable and interpretable convergence with graphs structures.

Related papers

• TopoFormer: Topology Meets Attention for Graph Learning [8.679678575739304]
  We introduce Topoformer, a framework for graph representation learning that encodes topological structure into attention-friendly sequences.<n>At the core of our method is Topo-Scan, a novel module that decomposes a graph into a short, ordered sequence of topological tokens.
  arXiv  Detail & Related papers  (2026-07-30T14:18:17Z)
• TopoAlign: Topology-Aware Visual Representation Alignment [18.80110544625249]
  TopoAlign is a framework for visually comparing model representations from a structural perspective.<n>We show that TopoAlign provides meaningful insights into representation structure and alignment from a topological perspective.
  arXiv  Detail & Related papers  (2026-05-25T07:58:26Z)
• Explaining Vision GNNs: A Semantic and Visual Analysis of Graph-based Image Classification [4.714421854862438]
  Graph Neural Networks (GNNs) have emerged as an efficient alternative to convolutional approaches for vision tasks.<n>Despite their efficiency, the explainability of GNN-based vision models remains underexplored.
  arXiv  Detail & Related papers  (2025-04-28T11:13:40Z)
• Weisfeiler and Lehman Go Paths: Learning Topological Features via Path Complexes [4.23480641508611]
  Graph Neural Networks (GNNs) are theoretically bounded by the 1-Weisfeiler-Lehman test. Our study presents a novel perspective by focusing on simple paths within graphs during the topological message-passing process.
  arXiv  Detail & Related papers  (2023-08-13T19:45:20Z)
• On the Expressivity of Persistent Homology in Graph Learning [13.608942872770855]
  Persistent homology, a technique from computational topology, has recently shown strong empirical performance in the context of graph classification.<n>This paper provides a brief introduction to persistent homology in the context of graphs, as well as a theoretical discussion and empirical analysis of its expressivity for graph learning tasks.
  arXiv  Detail & Related papers  (2023-02-20T08:19:19Z)
• Geometry Contrastive Learning on Heterogeneous Graphs [50.58523799455101]
  This paper proposes a novel self-supervised learning method, termed as Geometry Contrastive Learning (GCL) GCL views a heterogeneous graph from Euclidean and hyperbolic perspective simultaneously, aiming to make a strong merger of the ability of modeling rich semantics and complex structures. Extensive experiments on four benchmarks data sets show that the proposed approach outperforms the strong baselines.
  arXiv  Detail & Related papers  (2022-06-25T03:54:53Z)
• Duality-Induced Regularizer for Semantic Matching Knowledge Graph Embeddings [70.390286614242]
  We propose a novel regularizer -- namely, DUality-induced RegulArizer (DURA) -- which effectively encourages the entities with similar semantics to have similar embeddings. Experiments demonstrate that DURA consistently and significantly improves the performance of state-of-the-art semantic matching models.
  arXiv  Detail & Related papers  (2022-03-24T09:24:39Z)
• Learning Gaussian Graphical Models with Latent Confounders [74.72998362041088]
  We compare and contrast two strategies for inference in graphical models with latent confounders. While these two approaches have similar goals, they are motivated by different assumptions about confounding. We propose a new method, which combines the strengths of these two approaches.
  arXiv  Detail & Related papers  (2021-05-14T00:53:03Z)
• High-order Semantic Role Labeling [86.29371274587146]
  This paper introduces a high-order graph structure for the neural semantic role labeling model. It enables the model to explicitly consider not only the isolated predicate-argument pairs but also the interaction between the predicate-argument pairs. Experimental results on 7 languages of the CoNLL-2009 benchmark show that the high-order structural learning techniques are beneficial to the strong performing SRL models.
  arXiv  Detail & Related papers  (2020-10-09T15:33:54Z)
• Structural Landmarking and Interaction Modelling: on Resolution Dilemmas in Graph Classification [50.83222170524406]
  We study the intrinsic difficulty in graph classification under the unified concept of resolution dilemmas'' We propose SLIM'', an inductive neural network model for Structural Landmarking and Interaction Modelling.
  arXiv  Detail & Related papers  (2020-06-29T01:01:42Z)
• Embedding Graph Auto-Encoder for Graph Clustering [90.8576971748142]
  Graph auto-encoder (GAE) models are based on semi-supervised graph convolution networks (GCN) We design a specific GAE-based model for graph clustering to be consistent with the theory, namely Embedding Graph Auto-Encoder (EGAE) EGAE consists of one encoder and dual decoders.
  arXiv  Detail & Related papers  (2020-02-20T09:53:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.