Neural Isomorphic Fields: A Transformer-based Algebraic Numerical Embedding

• URL: http://arxiv.org/abs/2601.12095v1
• Date: Sat, 17 Jan 2026 16:25:52 GMT
• Title: Neural Isomorphic Fields: A Transformer-based Algebraic Numerical Embedding
• Abstract summary: We propose embedding vectors for numbers instead of directly using their raw values.<n>These embeddings aim to retain essential algebraic properties while preventing numerical instabilities.<n>We introduce, for the first time, a fixed-length number embedding vector that preserves algebraic operations.
• Score: 4.0782474409194736
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural network models often face challenges when processing very small or very large numbers due to issues such as overflow, underflow, and unstable output variations. To mitigate these problems, we propose using embedding vectors for numbers instead of directly using their raw values. These embeddings aim to retain essential algebraic properties while preventing numerical instabilities. In this paper, we introduce, for the first time, a fixed-length number embedding vector that preserves algebraic operations, including addition, multiplication, and comparison, within the field of rational numbers. We propose a novel Neural Isomorphic Field, a neural abstraction of algebraic structures such as groups and fields. The elements of this neural field are embedding vectors that maintain algebraic structure during computations. Our experiments demonstrate that addition performs exceptionally well, achieving over 95 percent accuracy on key algebraic tests such as identity, closure, and associativity. In contrast, multiplication exhibits challenges, with accuracy ranging from 53 percent to 73 percent across various algebraic properties. These findings highlight the model's strengths in preserving algebraic properties under addition while identifying avenues for further improvement in handling multiplication.

Related papers

• Grokking Finite-Dimensional Algebra [5.471648649900293]
  grokking refers to the sudden transition from a long memorization to generalization observed during neural networks training.<n>We show that grokking emerges naturally as models must learn discrete representations of algebraic elements.
  arXiv  Detail & Related papers  (2026-02-23T05:55:52Z)
• From Polynomials to Databases: Arithmetic Structures in Galois Theory [0.0]
  We develop a framework for classifying Galois groups of irreducible degree-7s over$mathbbQ$, combining explicit resolvent methods with machine learning techniques.<n>A database of over one million normalizedive project septics is constructed, each annotated with invariants$J_0, dots, J_4$ derived from binary transvections.
  arXiv  Detail & Related papers  (2025-11-20T18:29:38Z)
• A Fast Convoluted Story: Scaling Probabilistic Inference for Integer Arithmetic [4.7223923266180785]
  We formulate linear arithmetic over integer-valued random variables as tensor manipulations. We obtain a differentiable data structure, which unlocks, virtually for free, gradient-based learning.
  arXiv  Detail & Related papers  (2024-10-16T09:16:10Z)
• Knowledge Composition using Task Vectors with Learned Anisotropic Scaling [51.4661186662329]
  We introduce aTLAS, an algorithm that linearly combines parameter blocks with different learned coefficients, resulting in anisotropic scaling at the task vector level. We show that such linear combinations explicitly exploit the low intrinsicity of pre-trained models, with only a few coefficients being the learnable parameters. We demonstrate the effectiveness of our method in task arithmetic, few-shot recognition and test-time adaptation, with supervised or unsupervised objectives.
  arXiv  Detail & Related papers  (2024-07-03T07:54:08Z)
• Covering Number of Real Algebraic Varieties and Beyond: Improved Bounds and Applications [8.438718130535296]
  We prove upper bounds on the covering number of numerous sets in Euclidean space.<n>We illustrate the power of the result on three computational applications.
  arXiv  Detail & Related papers  (2023-11-09T03:06:59Z)
• Linear algebra with transformers [0.0]
  We show that transformers can be trained to perform numerical calculations with high accuracy. We consider problems of linear algebra: matrix transposition, addition, multiplication, eigenvalues and vectors, singular value decomposition, and inversion.
  arXiv  Detail & Related papers  (2021-12-03T13:21:57Z)
• Convolutional Filtering and Neural Networks with Non Commutative Algebras [153.20329791008095]
  We study the generalization of non commutative convolutional neural networks. We show that non commutative convolutional architectures can be stable to deformations on the space of operators.
  arXiv  Detail & Related papers  (2021-08-23T04:22:58Z)
• Recognizing and Verifying Mathematical Equations using Multiplicative Differential Neural Units [86.9207811656179]
  We show that memory-augmented neural networks (NNs) can achieve higher-order, memory-augmented extrapolation, stable performance, and faster convergence. Our models achieve a 1.53% average improvement over current state-of-the-art methods in equation verification and achieve a 2.22% Top-1 average accuracy and 2.96% Top-5 average accuracy for equation completion.
  arXiv  Detail & Related papers  (2021-04-07T03:50:11Z)
• Quantum-Inspired Algorithms from Randomized Numerical Linear Algebra [53.46106569419296]
  We create classical (non-quantum) dynamic data structures supporting queries for recommender systems and least-squares regression. We argue that the previous quantum-inspired algorithms for these problems are doing leverage or ridge-leverage score sampling in disguise.
  arXiv  Detail & Related papers  (2020-11-09T01:13:07Z)
• Algebraic Neural Networks: Stability to Deformations [179.55535781816343]
  We study algebraic neural networks (AlgNNs) with commutative algebras. AlgNNs unify diverse architectures such as Euclidean convolutional neural networks, graph neural networks, and group neural networks.
  arXiv  Detail & Related papers  (2020-09-03T03:41:38Z)
• Neural Arithmetic Units [84.65228064780744]
  Neural networks can approximate complex functions, but they struggle to perform exact arithmetic operations over real numbers. We present two new neural network components: the Neural Addition Unit (NAU), which can learn exact addition and subtraction, and the Neural multiplication Unit (NMU), which can multiply subsets of a vector. Our proposed units NAU and NMU, compared with previous neural units, converge more consistently, have fewer parameters, learn faster, can converge for larger hidden sizes, obtain sparse and meaningful weights, and can extrapolate to negative and small values.
  arXiv  Detail & Related papers  (2020-01-14T19:35:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.