Finding Meaning in Embeddings: Concept Separation Curves

• URL: http://arxiv.org/abs/2604.21555v1
• Date: Thu, 23 Apr 2026 11:29:58 GMT
• Title: Finding Meaning in Embeddings: Concept Separation Curves
• Abstract summary: Sentence embedding techniques aim to encode key concepts of a sentence's meaning in a vector space.<n>The majority of evaluation approaches for sentence embedding quality rely on the use of additional classifiers or downstream tasks.<n>We propose a novel method for evaluating the effectiveness of sentence embedding methods in capturing sentence-level concepts.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sentence embedding techniques aim to encode key concepts of a sentence's meaning in a vector space. However, the majority of evaluation approaches for sentence embedding quality rely on the use of additional classifiers or downstream tasks. These additional components make it unclear whether good results stem from the embedding itself or from the classifier's behaviour. In this paper, we propose a novel method for evaluating the effectiveness of sentence embedding methods in capturing sentence-level concepts. Our approach is classifier-independent, allowing for an objective assessment of the model's performance. The approach adopted in this study involves the systematic introduction of syntactic noise and semantic negations into sentences, with the subsequent quantification of their relative effects on the resulting embeddings. The visualisation of these effects is facilitated by Concept Separation Curves, which show the model's capacity to differentiate between conceptual and surface-level variations. By leveraging data from multiple domains, employing both Dutch and English languages, and examining sentence lengths, this study offers a compelling demonstration that Concept Separation Curves provide an interpretable, reproducible, and cross-model approach for evaluating the conceptual stability of sentence embeddings.

Related papers

• Studying Classifier(-Free) Guidance From a Classifier-Centric Perspective [100.54185280153753]
  We find that both classifier guidance and classifier-free guidance achieve conditional generation by pushing the denoising diffusion trajectories away from decision boundaries.<n>We propose a generic postprocessing step built upon flow-matching to shrink the gap between the learned distribution for a pretrained denoising diffusion model and the real data distribution.
  arXiv  Detail & Related papers  (2025-03-13T17:59:59Z)
• Scalable Multi-phase Word Embedding Using Conjunctive Propositional Clauses [9.691610260478248]
  We introduce a novel approach incorporating two-phase training to discover contextual embeddings of input sequences.<n>This technique not only facilitates the design of a scalable model but also preserves interpretability.<n>Our experimental findings revealed that the proposed method yields competitive performance compared to the previous approaches.
  arXiv  Detail & Related papers  (2025-01-31T10:39:04Z)
• Independence Constrained Disentangled Representation Learning from Epistemological Perspective [13.51102815877287]
  Disentangled Representation Learning aims to improve the explainability of deep learning methods by training a data encoder that identifies semantically meaningful latent variables in the data generation process. There is no consensus regarding the objective of disentangled representation learning. We propose a novel method for disentangled representation learning by employing an integration of mutual information constraint and independence constraint.
  arXiv  Detail & Related papers  (2024-09-04T13:00:59Z)
• DenoSent: A Denoising Objective for Self-Supervised Sentence Representation Learning [59.4644086610381]
  We propose a novel denoising objective that inherits from another perspective, i.e., the intra-sentence perspective. By introducing both discrete and continuous noise, we generate noisy sentences and then train our model to restore them to their original form. Our empirical evaluations demonstrate that this approach delivers competitive results on both semantic textual similarity (STS) and a wide range of transfer tasks.
  arXiv  Detail & Related papers  (2024-01-24T17:48:45Z)
• Posthoc Interpretation via Quantization [9.510336895838703]
  We introduce a new approach, called Posthoc Interpretation via Quantization (PIQ), for interpreting decisions made by trained classifiers. Our method utilizes vector quantization to transform the representations of a classifier into a discrete, class-specific latent space. Our model formulation also enables learning concepts by incorporating the supervision of pretrained annotation models.
  arXiv  Detail & Related papers  (2023-03-22T15:37:43Z)
• Resolving label uncertainty with implicit posterior models [71.62113762278963]
  We propose a method for jointly inferring labels across a collection of data samples. By implicitly assuming the existence of a generative model for which a differentiable predictor is the posterior, we derive a training objective that allows learning under weak beliefs.
  arXiv  Detail & Related papers  (2022-02-28T18:09:44Z)
• Discriminative Attribution from Counterfactuals [64.94009515033984]
  We present a method for neural network interpretability by combining feature attribution with counterfactual explanations. We show that this method can be used to quantitatively evaluate the performance of feature attribution methods in an objective manner.
  arXiv  Detail & Related papers  (2021-09-28T00:53:34Z)
• On Sample Based Explanation Methods for NLP:Efficiency, Faithfulness, and Semantic Evaluation [23.72825603188359]
  We can improve the interpretability of explanations by allowing arbitrary text sequences as the explanation unit. We propose a semantic-based evaluation metric that can better align with humans' judgment of explanations.
  arXiv  Detail & Related papers  (2021-06-09T00:49:56Z)
• On the Faithfulness Measurements for Model Interpretations [100.2730234575114]
  Post-hoc interpretations aim to uncover how natural language processing (NLP) models make predictions. To tackle these issues, we start with three criteria: the removal-based criterion, the sensitivity of interpretations, and the stability of interpretations. Motivated by the desideratum of these faithfulness notions, we introduce a new class of interpretation methods that adopt techniques from the adversarial domain.
  arXiv  Detail & Related papers  (2021-04-18T09:19:44Z)
• Weakly-Supervised Aspect-Based Sentiment Analysis via Joint Aspect-Sentiment Topic Embedding [71.2260967797055]
  We propose a weakly-supervised approach for aspect-based sentiment analysis. We learn sentiment, aspect> joint topic embeddings in the word embedding space. We then use neural models to generalize the word-level discriminative information.
  arXiv  Detail & Related papers  (2020-10-13T21:33:24Z)
• The Explanation Game: Towards Prediction Explainability through Sparse Communication [6.497816402045099]
  We provide a unified perspective of explainability as a problem between an explainer and a layperson. We use this framework to compare several prior approaches for extracting explanations. We propose new embedded methods for explainability, through the use of selective, sparse attention.
  arXiv  Detail & Related papers  (2020-04-28T22:27:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.