Toward a Functional Geometric Algebra for Natural Language Semantics

• URL: http://arxiv.org/abs/2604.25902v1
• Date: Tue, 28 Apr 2026 17:47:46 GMT
• Title: Toward a Functional Geometric Algebra for Natural Language Semantics
• Abstract summary: I argue that geometric algebra provides a mathematically superior foundation for semantic representation.<n>I develop the formal foundations, identify three core capabilities that GA provides and linear algebra does not.<n>I show how GA-based operations already implicit in current transformer architectures can be made explicit and extended.
• Score: 4.219950401172951
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Distributional and neural approaches to natural language semantics have been built almost exclusively on conventional linear algebra: vectors, matrices, tensors, and the operations that accompany them. These methods have achieved remarkable empirical success, yet they face persistent structural limitations in compositional semantics, type sensitivity, and interpretability. I argue in this paper that geometric algebra (GA) -- specifically, Clifford algebras -- provides a mathematically superior foundation for semantic representation, and that a Functional Geometric Algebra (FGA) framework extends GA toward a typed, compositional semantics capable of supporting inference, transformation, and interpretability while retaining full compatibility with distributional learning and modern neural architectures. I develop the formal foundations, identify three core capabilities that GA provides and linear algebra does not, present a detailed worked example illustrating operator-level semantic contrasts, and show how GA-based operations already implicit in current transformer architectures can be made explicit and extended. The central claim is not merely increased dimensionality but increased structural organization: GA expands an $n$-dimensional embedding space into a $2^n$ multivector algebra where base semantic concepts and their higher-order interactions are represented within a single, principled algebraic framework.

Related papers

• Are Common Substructures Transferable? Riemannian Graph Foundation Model with Neural Vector Bundles [53.65187206795911]
  We develop a graph intrinsic geometry learning framework called Neural Vector Bundle.<n>Building on this, we design GAUGE, a pretrainable neural architecture that constructs the vector bundle and flattens geometrically compatible local coordinates.<n>We empirically validate its superior expressiveness in challenging tasks including zero-shot link prediction and graph isomorphism.
  arXiv  Detail & Related papers  (2026-06-02T07:35:42Z)
• Grokking Finite-Dimensional Algebra [5.471648649900293]
  grokking refers to the sudden transition from a long memorization to generalization observed during neural networks training.<n>We show that grokking emerges naturally as models must learn discrete representations of algebraic elements.
  arXiv  Detail & Related papers  (2026-02-23T05:55:52Z)
• On the structural properties of Lie algebras via associated labeled directed graphs [0.0]
  We present a method for associating labeled directed graphs to finite-dimensional Lie algebras.<n>We analyze properties of valid graphs given the antisymmetry property of the Lie bracket as well as the Jacobi identity.<n>We develop graph-theoretic criteria for solvability, nilpotency, presence of ideals, simplicity, semisimplicity, and reductiveness of an algebra.
  arXiv  Detail & Related papers  (2026-01-22T18:09:16Z)
• Multiary gradings [0.0]
  We introduce the notion of grading by multiary groups and investigate various compatibility conditions between the arity of algebra operations and grading group operations.<n>The theory reveals fundamentally new phenomena not present in the binary case, such as the existence of higher power gradings and nontrivial constraints on arity compatibility.
  arXiv  Detail & Related papers  (2026-01-16T19:44:27Z)
• An Algebraic Representation Theorem for Linear GENEOs in Geometric Machine Learning [1.3425748364842416]
  Group Equivariant Non-Expansive Operators (GENEOs) have emerged as a powerful class of operators for encoding symmetries.<n>We introduce a new representation theorem for linear GENEOs acting between different perception pairs.<n>We also prove the compactness and convexity of the space of linear GENEOs.
  arXiv  Detail & Related papers  (2026-01-07T13:21:44Z)
• On Counts and Densities of Homogeneous Bent Functions: An Evolutionary Approach [60.00535100780336]
  This paper examines the use of Evolutionary Algorithms (EAs) to evolve homogeneous bent Boolean functions.<n>We introduce the notion of density of homogeneous bent functions, facilitating the algorithmic design that results in finding quadratic and cubic bent functions in different numbers of variables.
  arXiv  Detail & Related papers  (2025-11-16T15:33:40Z)
• Why Neural Network Can Discover Symbolic Structures with Gradient-based Training: An Algebraic and Geometric Foundation for Neurosymbolic Reasoning [73.18052192964349]
  We develop a theoretical framework that explains how discrete symbolic structures can emerge naturally from continuous neural network training dynamics.<n>By lifting neural parameters to a measure space and modeling training as Wasserstein gradient flow, we show that under geometric constraints, the parameter measure $mu_t$ undergoes two concurrent phenomena.
  arXiv  Detail & Related papers  (2025-06-26T22:40:30Z)
• Directional Non-Commutative Monoidal Structures for Compositional Embeddings in Machine Learning [0.0]
  We introduce a new structure for compositional embeddings built on directional non-commutative monoidal operators.<n>Our construction defines a distinct composition operator circ_i for each axis i, ensuring associative combination along each axis without imposing global commutativity.<n>All axis-specific operators commute with one another, enforcing a global interchange law that enables consistent crossaxis compositions.
  arXiv  Detail & Related papers  (2025-05-21T13:27:14Z)
• Algebras of actions in an agent's representations of the world [46.74201905814679]
  We use our framework to reproduce the symmetry-based representations from the symmetry-based disentangled representation learning formalism.<n>We then study the algebras of the transformations of worlds with features that occur in simple reinforcement learning scenarios.<n>Using computational methods, that we developed, we extract the algebras of the transformations of these worlds and classify them according to their properties.
  arXiv  Detail & Related papers  (2023-10-02T18:24:51Z)
• Geometry Interaction Knowledge Graph Embeddings [153.69745042757066]
  We propose Geometry Interaction knowledge graph Embeddings (GIE), which learns spatial structures interactively between the Euclidean, hyperbolic and hyperspherical spaces. Our proposed GIE can capture a richer set of relational information, model key inference patterns, and enable expressive semantic matching across entities.
  arXiv  Detail & Related papers  (2022-06-24T08:33:43Z)
• Learning Algebraic Representation for Systematic Generalization in Abstract Reasoning [109.21780441933164]
  We propose a hybrid approach to improve systematic generalization in reasoning. We showcase a prototype with algebraic representation for the abstract spatial-temporal task of Raven's Progressive Matrices (RPM) We show that the algebraic representation learned can be decoded by isomorphism to generate an answer.
  arXiv  Detail & Related papers  (2021-11-25T09:56:30Z)
• Learning Algebraic Recombination for Compositional Generalization [71.78771157219428]
  We propose LeAR, an end-to-end neural model to learn algebraic recombination for compositional generalization. Key insight is to model the semantic parsing task as a homomorphism between a latent syntactic algebra and a semantic algebra. Experiments on two realistic and comprehensive compositional generalization demonstrate the effectiveness of our model.
  arXiv  Detail & Related papers  (2021-07-14T07:23:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.