DySem: Uncovering Dynamic Semantic Components of Large Language Models for Calculating Semantic Textual Similarity

• URL: http://arxiv.org/abs/2605.29751v2
• Date: Fri, 29 May 2026 04:50:20 GMT
• Title: DySem: Uncovering Dynamic Semantic Components of Large Language Models for Calculating Semantic Textual Similarity
• Abstract summary: Calculating semantic textual similarity is a foundational task in natural language processing.<n>We propose DySem, a novel training-free framework that investigates more semantic-related internal components of large language models.
• Score: 12.354777054071379
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Calculating semantic textual similarity is a foundational task in natural language processing. Current large language models (LLMs) based methods typically rely on extracting last-layer hidden states with fixed dimensions to compute similarity for every text pairs. We argue that this paradigm is suffer from two limitations: (i) The last hidden layer encodes more general knowledge rather than just semantic knowledge, making it suboptimal for semantic similarity computation; (ii) The hidden layer dimensions of LLMs are generally very large, which introduces some redundancy and noise for representing semantics. In this work, we propose DySem, a novel training-free framework that investigates more semantic-related internal components of LLMs via multilingual consensus, and shifts away from static representation spaces in favor of dynamic, sample-specific semantic dimensions by constructing text-dependent joint semantic set and computes similarity over this shared dimensional subset. Extensive experiments across various LLMs show that our method consistently outperforms recent baselines while maintaining lower dimensions for similarity calculation. The code is released at https://github.com/szu-tera/DySem.

Related papers

• GRACE: LLM-Grounded Semantic Metric Spaces for Scalable Mixed-Data Clustering [19.352298911016668]
  GRACE is a framework for scalable mixed-data clustering.<n>It maps heterogeneous values into knowledge-informed descriptions.<n>It achieves superior clustering accuracy and conceptual interpretability over 11 competing methods.
  arXiv  Detail & Related papers  (2026-08-08T03:14:50Z)
• Scalable Semantic Steering of Embedding Projections [7.516736502809871]
  Low-dimensional projections support interactive visual analysis of high-dimensional data embeddings, but their structure often does not align with analyst-defined semantic relationships.<n>Recent LLM-augmented semantic steering methods address this gap by externalizing analyst intent from user-defined groups of seed examples.<n>We propose a scalable semantic steering method that shifts semantic computation from individual items to user-defined groups.
  arXiv  Detail & Related papers  (2026-07-04T18:24:00Z)
• Semantic Tree Inference on Text Corpa using a Nested Density Approach together with Large Language Model Embeddings [0.0]
  We propose a nested density clustering approach to infer hierarchical trees of semantically related texts.<n>By embedding dense clusters into increasingly diffuse ones, we construct a tree structure that captures hierarchical semantic relationships among texts.
  arXiv  Detail & Related papers  (2025-12-29T13:55:23Z)
• Semantic-KG: Using Knowledge Graphs to Construct Benchmarks for Measuring Semantic Similarity [42.873412319680035]
  This paper introduces a novel method for generating benchmarks to evaluate semantic similarity methods for Large Language Models outputs.<n>We generate benchmark datasets in four different domains (general knowledge, biomedicine, finance, biology)<n>We observe that the sub-type of semantic variation, as well as the domain of the benchmark impact the performance of semantic similarity methods.
  arXiv  Detail & Related papers  (2025-11-25T05:07:08Z)
• QUDsim: Quantifying Discourse Similarities in LLM-Generated Text [70.22275200293964]
  We introduce an abstraction based on linguistic theories in Questions Under Discussion (QUD) and question semantics to help quantify differences in discourse progression.<n>We then use this framework to build $textbfQUDsim$, a similarity metric that can detect discursive parallels between documents.<n>Using QUDsim, we find that LLMs often reuse discourse structures (more so than humans) across samples, even when content differs.
  arXiv  Detail & Related papers  (2025-04-12T23:46:09Z)
• Tomato, Tomahto, Tomate: Measuring the Role of Shared Semantics among Subwords in Multilingual Language Models [88.07940818022468]
  We take an initial step on measuring the role of shared semantics among subwords in the encoder-only multilingual language models (mLMs) We form "semantic tokens" by merging the semantically similar subwords and their embeddings. inspections on the grouped subwords show that they exhibit a wide range of semantic similarities.
  arXiv  Detail & Related papers  (2024-11-07T08:38:32Z)
• Spatial Semantic Recurrent Mining for Referring Image Segmentation [63.34997546393106]
  We propose Stextsuperscript2RM to achieve high-quality cross-modality fusion. It follows a working strategy of trilogy: distributing language feature, spatial semantic recurrent coparsing, and parsed-semantic balancing. Our proposed method performs favorably against other state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2024-05-15T00:17:48Z)
• SemGrasp: Semantic Grasp Generation via Language Aligned Discretization [53.43801984965309]
  This paper presents a novel semantic-based grasp generation method, termed SemGrasp. We introduce a discrete representation that aligns the grasp space with semantic space, enabling the generation of grasp postures. A Multimodal Large Language Model (MLLM) is subsequently fine-tuned, integrating object, grasp, and language within a unified semantic space.
  arXiv  Detail & Related papers  (2024-04-04T16:58:26Z)
• Breaking Down Word Semantics from Pre-trained Language Models through Layer-wise Dimension Selection [0.0]
  This paper aims to disentangle semantic sense from BERT by applying a binary mask to middle outputs across the layers. The disentangled embeddings are evaluated through binary classification to determine if the target word in two different sentences has the same meaning.
  arXiv  Detail & Related papers  (2023-10-08T11:07:19Z)
• Multi-Semantic Interactive Learning for Object Detection [11.392255321898324]
  We propose multi-semantic interactive learning (MSIL) to mine the semantic relevance between different branches. MSIL can be integrated into existing object detection nets as a plug-and-play component.
  arXiv  Detail & Related papers  (2023-03-18T13:14:15Z)
• Integrating Language Guidance into Vision-based Deep Metric Learning [78.18860829585182]
  We propose to learn metric spaces which encode semantic similarities as embedding space. These spaces should be transferable to classes beyond those seen during training. This causes learned embedding spaces to encode incomplete semantic context and misrepresent the semantic relation between classes.
  arXiv  Detail & Related papers  (2022-03-16T11:06:50Z)
• Contextualized Semantic Distance between Highly Overlapped Texts [85.1541170468617]
  Overlapping frequently occurs in paired texts in natural language processing tasks like text editing and semantic similarity evaluation. This paper aims to address the issue with a mask-and-predict strategy. We take the words in the longest common sequence as neighboring words and use masked language modeling (MLM) to predict the distributions on their positions. Experiments on Semantic Textual Similarity show NDD to be more sensitive to various semantic differences, especially on highly overlapped paired texts.
  arXiv  Detail & Related papers  (2021-10-04T03:59:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.