The Geometry of Categorical and Hierarchical Concepts in Large Language Models
- URL: http://arxiv.org/abs/2406.01506v1
- Date: Mon, 3 Jun 2024 16:34:01 GMT
- Title: The Geometry of Categorical and Hierarchical Concepts in Large Language Models
- Authors: Kiho Park, Yo Joong Choe, Yibo Jiang, Victor Veitch,
- Abstract summary: We study the two foundational questions in this area.
How are categorical concepts, such as'mammal', 'bird','reptile', 'fish', represented?
For example, how is the fact that 'dog' is a kind of'mammal' encoded?
- Score: 15.126806053878855
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Understanding how semantic meaning is encoded in the representation spaces of large language models is a fundamental problem in interpretability. In this paper, we study the two foundational questions in this area. First, how are categorical concepts, such as {'mammal', 'bird', 'reptile', 'fish'}, represented? Second, how are hierarchical relations between concepts encoded? For example, how is the fact that 'dog' is a kind of 'mammal' encoded? We show how to extend the linear representation hypothesis to answer these questions. We find a remarkably simple structure: simple categorical concepts are represented as simplices, hierarchically related concepts are orthogonal in a sense we make precise, and (in consequence) complex concepts are represented as polytopes constructed from direct sums of simplices, reflecting the hierarchical structure. We validate these theoretical results on the Gemma large language model, estimating representations for 957 hierarchically related concepts using data from WordNet.
Related papers
- On the Origins of Linear Representations in Large Language Models [51.88404605700344]
We introduce a simple latent variable model to formalize the concept dynamics of the next token prediction.
Experiments show that linear representations emerge when learning from data matching the latent variable model.
We additionally confirm some predictions of the theory using the LLaMA-2 large language model.
arXiv Detail & Related papers (2024-03-06T17:17:36Z) - The Linear Representation Hypothesis and the Geometry of Large Language Models [12.387530469788738]
Informally, the 'linear representation hypothesis' is the idea that high-level concepts are represented linearly as directions in some representation space.
This paper addresses two closely related questions: What does "linear representation" actually mean?
We show how to unify all notions of linear representation using counterfactual pairs.
arXiv Detail & Related papers (2023-11-07T01:59:11Z) - Concept2Box: Joint Geometric Embeddings for Learning Two-View Knowledge
Graphs [77.10299848546717]
Concept2Box is a novel approach that jointly embeds the two views of a KG.
Box embeddings learn the hierarchy structure and complex relations such as overlap and disjoint among them.
We propose a novel vector-to-box distance metric and learn both embeddings jointly.
arXiv Detail & Related papers (2023-07-04T21:37:39Z) - How Do Transformers Learn Topic Structure: Towards a Mechanistic
Understanding [56.222097640468306]
We provide mechanistic understanding of how transformers learn "semantic structure"
We show, through a combination of mathematical analysis and experiments on Wikipedia data, that the embedding layer and the self-attention layer encode the topical structure.
arXiv Detail & Related papers (2023-03-07T21:42:17Z) - Succinct Representations for Concepts [12.134564449202708]
Foundation models like chatGPT have demonstrated remarkable performance on various tasks.
However, for many questions, they may produce false answers that look accurate.
In this paper, we introduce succinct representations of concepts based on category theory.
arXiv Detail & Related papers (2023-03-01T12:11:23Z) - Concept Algebra for (Score-Based) Text-Controlled Generative Models [27.725860408234478]
This paper concerns the structure of learned representations in text-guided generative models.
A key property of such models is that they can compose disparate concepts in a disentangled' manner.
Here, we focus on the idea that concepts are encoded as subspaces of some representation space.
arXiv Detail & Related papers (2023-02-07T20:43:48Z) - On the Complexity of Bayesian Generalization [141.21610899086392]
We consider concept generalization at a large scale in the diverse and natural visual spectrum.
We study two modes when the problem space scales up, and the $complexity$ of concepts becomes diverse.
arXiv Detail & Related papers (2022-11-20T17:21:37Z) - Latent Topology Induction for Understanding Contextualized
Representations [84.7918739062235]
We study the representation space of contextualized embeddings and gain insight into the hidden topology of large language models.
We show there exists a network of latent states that summarize linguistic properties of contextualized representations.
arXiv Detail & Related papers (2022-06-03T11:22:48Z) - Visual Superordinate Abstraction for Robust Concept Learning [80.15940996821541]
Concept learning constructs visual representations that are connected to linguistic semantics.
We ascribe the bottleneck to a failure of exploring the intrinsic semantic hierarchy of visual concepts.
We propose a visual superordinate abstraction framework for explicitly modeling semantic-aware visual subspaces.
arXiv Detail & Related papers (2022-05-28T14:27:38Z) - Towards Visual Semantics [17.1623244298824]
We study how humans build mental representations, i.e., concepts, of what they visually perceive.
In this paper we provide a theory and an algorithm which learns substance concepts which correspond to the concepts, that we call classification concepts.
The experiments, though preliminary, show that the algorithm manages to acquire the notions of Genus and Differentia with reasonable accuracy.
arXiv Detail & Related papers (2021-04-26T07:28:02Z) - The ERA of FOLE: Foundation [0.0]
This paper continues the discussion of the representation and interpretation of the first-order logical environment ttfamily FOLE.
The formalism and semantics of (many-sorted) first-order logic can be developed in both a emphclassification form and an emphinterpretation form.
In general, the ttfamily FOLE representation uses a conceptual approach, that is completely compatible with the theory of institutions, formal concept analysis and information flow.
arXiv Detail & Related papers (2015-12-23T11:00:15Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.