Internalizing Tools as Morphisms in Graded Transformers

• URL: http://arxiv.org/abs/2511.17840v1
• Date: Fri, 21 Nov 2025 23:27:53 GMT
• Title: Internalizing Tools as Morphisms in Graded Transformers
• Abstract summary: We introduce a graded formulation of internal symbolic computation for transformers.<n>The hidden space is endowed with a grading $V=bigoplus_gin GV_g$, and symbolic operations are realized as typed block maps.<n>A self-supervised emphgraded utility functional governs activation and yields sparse, interpretable behavior.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce a graded formulation of internal symbolic computation for transformers. The hidden space is endowed with a grading $V=\bigoplus_{g\in G}V_g$, and symbolic operations are realized as typed block maps (morphisms) $φ_{h\leftarrow g}:V_g\to V_h$ that are activated selectively by a differentiable routing policy. A self-supervised \emph{graded utility functional}, defined as the loss reduction induced by a candidate morphism, governs activation and yields sparse, interpretable behavior. We develop the algebraic and geometric foundations: an internal model category whose objects are homogeneous components and whose morphisms are admissible grade transitions; adjoint pairs encoding typed round trips; and information-geometric interpretations in terms of KL gain, mirror descent with Bregman divergences, and Fisher natural gradients. Methodologically, we specify a utility--aware routing mechanism and objective that remain fully end-to-end differentiable. Analytic case studies and lightweight sanity checks illustrate selective morphic activation on hybrid symbolic-linguistic tasks. The framework unifies symbolic computation, geometry, and self--supervised learning within the \emph{graded transformer} formalism \cite{sh-89,sh-95}, while subsuming prior external-tool paradigms (e.g., Toolformer \cite{toolformer2023}) as a special case via functorial internalization.

Related papers

• Hierarchical Grading in Large Language Models [0.0]
  We introduce Graded Large Language Models (GLLMs)<n>GLLMs are a framework that equips the representation space of a transformer with a grading.<n>The construction extends the theory of graded neural networks and graded transformers to autoregressive language models.
  arXiv  Detail & Related papers  (2026-07-23T20:14:48Z)
• LAST: Bridging Vision-Language and Action Manifolds via Gromov-Wasserstein Alignment [71.3712075841632]
  LAST reconstructs the action space to establish local metric compatibility with the Vision-Language-Action modality.<n> LAST enables VLA models with superior convergence and generalizability.
  arXiv  Detail & Related papers  (2026-05-27T03:48:10Z)
• Function graph transformers universally approximate operators between function spaces [14.886643480836817]
  We study the approximation of nonlinear operators between function spaces by transformers.<n>Our approach is to lift functions to measures supported on their graphs and leverage a recently introduced measure-theoretic view of transformers.<n>We show that this framework models discretization refinement via convergence measures and provides a natural setting for operator learning.
  arXiv  Detail & Related papers  (2026-05-18T07:25:14Z)
• Geometric Factual Recall in Transformers [57.48371649045765]
  A common view casts internal weight matrices as associative memories over pairs of embeddings, requiring memorization counts that scale linearly with the number of facts.<n>We develop a theoretical and empirical account of an alternative, emphgeometric form of memorization in which learned embeddings encode structure directly.<n>In a controlled setting where a single-layer transformer must memorize random vectors from subjects to a shared attribute set, we prove that a logarithmic embedding dimension suffices.<n>We extend these results to the multi-hop setting, providing constructions with and without chain-of-thought that exhibit a provable
  arXiv  Detail & Related papers  (2026-05-12T17:22:22Z)
• Layerwise Dynamics for In-Context Classification in Transformers [27.052342498221748]
  We study multi-class linear classification in the hard no-margin regime.<n>We make computation identifiable by enforcing feature- and label-permutation equivariance at every layer.<n>This enables interpretability while maintaining functional equivalence.
  arXiv  Detail & Related papers  (2026-04-13T15:20:41Z)
• Coalgebras for categorical deep learning: Representability and universal approximation [0.0]
  We develop a coalgebraic foundation for equivariant representation in deep learning.<n>We establish a universal approximation theorem for equivariant maps in a generalized setting.<n>This work provides a categorical bridge between the abstract specification of invariant behavior and its concrete realization in neural architectures.
  arXiv  Detail & Related papers  (2026-03-03T18:18:50Z)
• Beyond Optimization: Intelligence as Metric-Topology Factorization under Geometric Incompleteness [6.0044467881527614]
  We argue that intelligence is not navigation through a fixed maze, but the ability to reshape representational geometry so desired behaviors become stable attractors.<n>We show any fixed metric is geometrically incomplete: for any local metric representation, some topological transformations make it singular or incoherent.<n>We introduce the Topological Urysohn Machine (TUM), implementing MTF through memory-amortized metric inference.
  arXiv  Detail & Related papers  (2026-02-08T13:59:22Z)
• Beyond Softmax: A Natural Parameterization for Categorical Random Variables [61.709831225296305]
  We introduce the $textitcatnat$ function, a function composed of a sequence of hierarchical binary splits.<n>A rich set of experiments show that the proposed function improves the learning efficiency and yields models characterized by consistently higher test performance.
  arXiv  Detail & Related papers  (2025-09-29T12:55:50Z)
• Why Neural Network Can Discover Symbolic Structures with Gradient-based Training: An Algebraic and Geometric Foundation for Neurosymbolic Reasoning [73.18052192964349]
  We develop a theoretical framework that explains how discrete symbolic structures can emerge naturally from continuous neural network training dynamics.<n>By lifting neural parameters to a measure space and modeling training as Wasserstein gradient flow, we show that under geometric constraints, the parameter measure $mu_t$ undergoes two concurrent phenomena.
  arXiv  Detail & Related papers  (2025-06-26T22:40:30Z)
• Alpay Algebra II: Identity as Fixed-Point Emergence in Categorical Data [0.0]
  I define identity as a fixed point that emerges through categorical recursion.<n>I prove the existence and uniqueness of such identity--fixed via ordinal-indexed iteration.<n>This paper positions identity as a mathematical structure that arises from within the logic of change itself computable, convergent, and categorically intrinsic.
  arXiv  Detail & Related papers  (2025-05-23T05:15:34Z)
• Symmetry-Breaking Descent for Invariant Cost Functionals [0.0]
  We study the problem of reducing a task cost functional $W : Hs(M) to mathbbR$, not assumed continuous or differentiable.<n>We show that symmetry-breaking deformations of the signal can reduce the cost.
  arXiv  Detail & Related papers  (2025-05-19T15:06:31Z)
• Discovering Physics Laws of Dynamical Systems via Invariant Function Learning [51.84691955495693]
  We consider learning underlying laws of dynamical systems governed by ordinary differential equations (ODE)<n>We propose a new method, known as textbfDisentanglement of textbfInvariant textbfFunctions (DIF)<n>Our code has been released as part of the AIRS library.
  arXiv  Detail & Related papers  (2025-02-06T20:46:50Z)
• Self-Attention as a Parametric Endofunctor: A Categorical Framework for Transformer Architectures [0.0]
  We develop a category-theoretic framework focusing on the linear components of self-attention.<n>We show that the query, key, and value maps naturally define a parametric 1-morphism in the 2-category $mathbfPara(Vect)$.<n> stacking multiple self-attention layers corresponds to constructing the free monad on this endofunctor.
  arXiv  Detail & Related papers  (2025-01-06T11:14:18Z)
• Quasi-symbolic Semantic Geometry over Transformer-based Variational AutoEncoder [10.880057430629126]
  We frame the sentence semantics as the composition of textitsemantic role - word content features and propose the formal semantic geometry.<n>We propose a new probing algorithm to guide the movement of sentence vectors over such geometry.<n> Experimental results reveal that the formal semantic geometry can potentially deliver better control and interpretation to sentence generation.
  arXiv  Detail & Related papers  (2022-10-12T14:20:33Z)
• Self-Supervised Learning Disentangled Group Representation as Feature [82.07737719232972]
  We show that existing Self-Supervised Learning (SSL) only disentangles simple augmentation features such as rotation and colorization. We propose an iterative SSL algorithm: Iterative Partition-based Invariant Risk Minimization (IP-IRM) We prove that IP-IRM converges to a fully disentangled representation and show its effectiveness on various benchmarks.
  arXiv  Detail & Related papers  (2021-10-28T16:12:33Z)
• Frame Averaging for Invariant and Equivariant Network Design [50.87023773850824]
  We introduce Frame Averaging (FA), a framework for adapting known (backbone) architectures to become invariant or equivariant to new symmetry types. We show that FA-based models have maximal expressive power in a broad setting. We propose a new class of universal Graph Neural Networks (GNNs), universal Euclidean motion invariant point cloud networks, and Euclidean motion invariant Message Passing (MP) GNNs.
  arXiv  Detail & Related papers  (2021-10-07T11:05:23Z)
• Convolutional Filtering and Neural Networks with Non Commutative Algebras [153.20329791008095]
  We study the generalization of non commutative convolutional neural networks. We show that non commutative convolutional architectures can be stable to deformations on the space of operators.
  arXiv  Detail & Related papers  (2021-08-23T04:22:58Z)
• A Differentiable Relaxation of Graph Segmentation and Alignment for AMR Parsing [75.36126971685034]
  We treat alignment and segmentation as latent variables in our model and induce them as part of end-to-end training. Our method also approaches that of a model that relies on citetLyu2018AMRPA's segmentation rules, which were hand-crafted to handle individual AMR constructions.
  arXiv  Detail & Related papers  (2020-10-23T21:22:50Z)
• Algebraic Neural Networks: Stability to Deformations [179.55535781816343]
  We study algebraic neural networks (AlgNNs) with commutative algebras. AlgNNs unify diverse architectures such as Euclidean convolutional neural networks, graph neural networks, and group neural networks.
  arXiv  Detail & Related papers  (2020-09-03T03:41:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.