Discovering Semantic Subdimensions through Disentangled Conceptual Representations

• URL: http://arxiv.org/abs/2508.21436v2
• Date: Fri, 19 Sep 2025 07:46:25 GMT
• Title: Discovering Semantic Subdimensions through Disentangled Conceptual Representations
• Authors: Yunhao Zhang, Shaonan Wang, Nan Lin, Xinyi Dong, Chong Li, Chengqing Zong,
• Abstract summary: This paper proposes a novel framework to investigate the subdimensions underlying coarse-grained semantic dimensions.<n>We introduce a Disentangled Continuous Representation Model (DCSRM) that decomposes word embeddings from large language models into multiple sub-embeddings.<n>Using these sub-embeddings, we identify a set of interpretable semantic subdimensions.<n>Our work offers more fine-grained interpretable semantic subdimensions of conceptual meaning.
• Score: 38.66662397064128
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Understanding the core dimensions of conceptual semantics is fundamental to uncovering how meaning is organized in language and the brain. Existing approaches often rely on predefined semantic dimensions that offer only broad representations, overlooking finer conceptual distinctions. This paper proposes a novel framework to investigate the subdimensions underlying coarse-grained semantic dimensions. Specifically, we introduce a Disentangled Continuous Semantic Representation Model (DCSRM) that decomposes word embeddings from large language models into multiple sub-embeddings, each encoding specific semantic information. Using these sub-embeddings, we identify a set of interpretable semantic subdimensions. To assess their neural plausibility, we apply voxel-wise encoding models to map these subdimensions to brain activation. Our work offers more fine-grained interpretable semantic subdimensions of conceptual meaning. Further analyses reveal that semantic dimensions are structured according to distinct principles, with polarity emerging as a key factor driving their decomposition into subdimensions. The neural correlates of the identified subdimensions support their cognitive and neuroscientific plausibility.

Related papers

• Interpretation as Linear Transformation: A Cognitive-Geometric Model of Belief and Meaning [0.0]
  I show how belief distortion, motivational drift, counterfactual evaluation, and the limits of mutual understanding arise from purely algebraic constraints.<n>I argue that this cognitive-geometric perspective clarifies the boundaries of influence in both human and artificial systems.
  arXiv  Detail & Related papers  (2025-12-10T17:13:01Z)
• On the Geometry of Semantics in Next-token Prediction [27.33243506775655]
  Modern language models capture linguistic meaning despite being trained solely through next-token prediction.<n>We investigate how this conceptually simple training objective leads models to extract and encode latent semantic and grammatical concepts.<n>Our work bridges distributional semantics, neural collapse geometry, and neural network training dynamics, providing insights into how NTP's implicit biases shape the emergence of meaning representations in language models.
  arXiv  Detail & Related papers  (2025-05-13T08:46:04Z)
• Identifying and interpreting non-aligned human conceptual representations using language modeling [0.0]
  We show that congenital blindness induces conceptual reorganization in both a-modal and sensory-related verbal domains. We find that blind individuals more strongly associate social and cognitive meanings to verbs related to motion. For some verbs, representations of blind and sighted are highly similar.
  arXiv  Detail & Related papers  (2024-03-10T13:02:27Z)
• Domain Embeddings for Generating Complex Descriptions of Concepts in Italian Language [65.268245109828]
  We propose a Distributional Semantic resource enriched with linguistic and lexical information extracted from electronic dictionaries. The resource comprises 21 domain-specific matrices, one comprehensive matrix, and a Graphical User Interface. Our model facilitates the generation of reasoned semantic descriptions of concepts by selecting matrices directly associated with concrete conceptual knowledge.
  arXiv  Detail & Related papers  (2024-02-26T15:04:35Z)
• Understanding Distributed Representations of Concepts in Deep Neural Networks without Supervision [25.449397570387802]
  We propose an unsupervised method for discovering distributed representations of concepts by selecting a principal subset of neurons. Our empirical findings demonstrate that instances with similar neuron activation states tend to share coherent concepts. It can be utilized to identify unlabeled subclasses within data and to detect the causes of misclassifications.
  arXiv  Detail & Related papers  (2023-12-28T07:33:51Z)
• A Geometric Notion of Causal Probing [85.49839090913515]
  The linear subspace hypothesis states that, in a language model's representation space, all information about a concept such as verbal number is encoded in a linear subspace.<n>We give a set of intrinsic criteria which characterize an ideal linear concept subspace.<n>We find that, for at least one concept across two languages models, the concept subspace can be used to manipulate the concept value of the generated word with precision.
  arXiv  Detail & Related papers  (2023-07-27T17:57:57Z)
• Hierarchical Semantic Tree Concept Whitening for Interpretable Image Classification [19.306487616731765]
  Post-hoc analysis can only discover the patterns or rules that naturally exist in models. We proactively instill knowledge to alter the representation of human-understandable concepts in hidden layers. Our method improves model interpretability, showing better disentanglement of semantic concepts, without negatively affecting model classification performance.
  arXiv  Detail & Related papers  (2023-07-10T04:54:05Z)
• Agentivit\`a e telicit\`a in GilBERTo: implicazioni cognitive [77.71680953280436]
  The goal of this study is to investigate whether a Transformer-based neural language model infers lexical semantics. The semantic properties considered are telicity (also combined with definiteness) and agentivity.
  arXiv  Detail & Related papers  (2023-07-06T10:52:22Z)
• Visual representations in the human brain are aligned with large language models [7.779248296336383]
  We show that large language models (LLMs) are beneficial for modelling the complex visual information extracted by the brain from natural scenes. We then train deep neural network models to transform image inputs into LLM representations.
  arXiv  Detail & Related papers  (2022-09-23T17:34:33Z)
• Visual Superordinate Abstraction for Robust Concept Learning [80.15940996821541]
  Concept learning constructs visual representations that are connected to linguistic semantics. We ascribe the bottleneck to a failure of exploring the intrinsic semantic hierarchy of visual concepts. We propose a visual superordinate abstraction framework for explicitly modeling semantic-aware visual subspaces.
  arXiv  Detail & Related papers  (2022-05-28T14:27:38Z)
• Towards Visual Semantics [17.1623244298824]
  We study how humans build mental representations, i.e., concepts, of what they visually perceive. In this paper we provide a theory and an algorithm which learns substance concepts which correspond to the concepts, that we call classification concepts. The experiments, though preliminary, show that the algorithm manages to acquire the notions of Genus and Differentia with reasonable accuracy.
  arXiv  Detail & Related papers  (2021-04-26T07:28:02Z)
• Closed-Form Factorization of Latent Semantics in GANs [65.42778970898534]
  A rich set of interpretable dimensions has been shown to emerge in the latent space of the Generative Adversarial Networks (GANs) trained for synthesizing images. In this work, we examine the internal representation learned by GANs to reveal the underlying variation factors in an unsupervised manner. We propose a closed-form factorization algorithm for latent semantic discovery by directly decomposing the pre-trained weights.
  arXiv  Detail & Related papers  (2020-07-13T18:05:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.