Numbers Already Carry Their Own Embeddings

• URL: http://arxiv.org/abs/2606.14108v1
• Date: Fri, 12 Jun 2026 04:41:51 GMT
• Title: Numbers Already Carry Their Own Embeddings
• Abstract summary: We introduce Adelic operation-preserved embeddings (AOE)<n>AOE captures both a number's real value and its modular (p-adic) signatures.<n>It delivers consistent gains including the first-ever perfect accuracy on the Weaving Pattern task.
• Score: 6.003090616009607
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce Adelic operation-preserved embeddings (AOE), a training-free representation that captures both a number's real value and its modular (p-adic) signatures. This construction preserves additive and multiplicative structure by design, turning numerical input into embeddings that "speak in the language of mathematics." Unlike prior approaches that rely on task-specific retraining, AOE is plug-and-play and drops seamlessly into existing architectures. On algebraic combinatorics benchmarks, it delivers consistent gains including the first-ever perfect accuracy on the Weaving Pattern task-while suggesting a principled path forward for overcoming the long-standing "number problem" in AI.

Related papers

• DiffMath: Symbol- and Graph-Aware Latent Diffusion Transformer for Handwritten Mathematical Expression Generation [47.67907704295109]
  Handwritten Mathematical Expression Generation (HMEG) is challenging due to the complex two-dimensional layouts and long-range dependencies.<n>In this work, we propose DiffMath, a symbol-oriented graph-aware latent diffusion framework.<n> Experiments show that DiffMath produces structurally consistent handwritten expressions, superior performance over existing methods, and improves the accuracy of downstream models.
  arXiv  Detail & Related papers  (2026-06-18T08:37:02Z)
• An Algebraic View of the Expressivity of Recurrent Language Models [50.88598486616582]
  The paper develops a unified algebraic account of the expressivity of recurrent neural networks.<n>The same architecture cannot implement an even-modulus counter once floating-point recurrences are enforced, yet realizes every even-modulus counter under unsigned-integer quantization.
  arXiv  Detail & Related papers  (2026-06-01T06:47:58Z)
• Value-Aware Numerical Representations for Transformer Language Models [1.2680800636608986]
  Transformer-based language models often achieve strong results on mathematical reasoning benchmarks.<n>A central limitation is that numbers are processed as symbolic tokens whose embeddings do not explicitly encode numerical value.<n>We introduce a value-aware numerical representation that augments standard tokenized inputs with a dedicated prefix token.
  arXiv  Detail & Related papers  (2026-01-14T18:59:14Z)
• Laying Anchors: Semantically Priming Numerals in Language Modeling [11.831883526217942]
  We introduce strategies to semantically prime numerals in any corpus by generating anchors governed by the distribution of numerals in said corpus. We demonstrate significant improvements in the mathematical grounding of our learned embeddings.
  arXiv  Detail & Related papers  (2024-04-02T00:02:00Z)
• Reverse That Number! Decoding Order Matters in Arithmetic Learning [49.5504492920404]
  Our work introduces a novel strategy that reevaluates the digit order by prioritizing output from the least significant digit. Compared to the previous state-of-the-art (SOTA) method, our findings reveal an overall improvement of in accuracy while requiring only a third of the tokens typically used during training.
  arXiv  Detail & Related papers  (2024-03-09T09:04:53Z)
• A Hybrid System for Systematic Generalization in Simple Arithmetic Problems [70.91780996370326]
  We propose a hybrid system capable of solving arithmetic problems that require compositional and systematic reasoning over sequences of symbols. We show that the proposed system can accurately solve nested arithmetical expressions even when trained only on a subset including the simplest cases.
  arXiv  Detail & Related papers  (2023-06-29T18:35:41Z)
• Tokenization Consistency Matters for Generative Models on Extractive NLP Tasks [54.306234256074255]
  We identify the issue of tokenization inconsistency that is commonly neglected in training generative models. This issue damages the extractive nature of these tasks after the input and output are tokenized inconsistently. We show that, with consistent tokenization, the model performs better in both in-domain and out-of-domain datasets.
  arXiv  Detail & Related papers  (2022-12-19T23:33:21Z)
• ELASTIC: Numerical Reasoning with Adaptive Symbolic Compiler [5.753951795185978]
  We introduce numEricaL reASoning with adapTive symbolIc Compiler (ELASTIC) model. ELASTIC is robust when conducting complicated reasoning. Experiments show that ELASTIC achieves 68.96 and 65.21 of execution accuracy and program accuracy.
  arXiv  Detail & Related papers  (2022-10-18T19:04:19Z)
• Arithmetic-Based Pretraining -- Improving Numeracy of Pretrained Language Models [67.48894919842576]
  State-of-the-art pretrained language models tend to perform below their capabilities when applied out-of-the-box on tasks that require numeracy. We propose a new extended pretraining approach called Arithmetic-Based Pretraining that jointly addresses both in one extended pretraining step. Our experiments show the effectiveness of Arithmetic-Based Pretraining in three different tasks that require improved numeracy.
  arXiv  Detail & Related papers  (2022-05-13T16:10:13Z)
• Non-determinsitic algebraic rewriting as adjunction [0.0]
  We develop a general model theoretic semantics to rewriting beyond the usual confluence and termination assumptions. This is based on preordered algebra which is a model theory that extends many sorted algebra.
  arXiv  Detail & Related papers  (2022-04-26T07:55:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.