Probing Structural Mathematical Reasoning in Language Models with Algebraic Trapdoors

• URL: http://arxiv.org/abs/2605.04352v1
• Date: Tue, 05 May 2026 23:16:30 GMT
• Title: Probing Structural Mathematical Reasoning in Language Models with Algebraic Trapdoors
• Abstract summary: We introduce a benchmark suite for evaluating structural mathematical reasoning in language models.<n>Each instance presents a finitely generated subgroup as a list of integer matrices.<n>We report empirical results across five representative reasoning traces from two state-of-the-art models.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce a benchmark suite for evaluating structural mathematical reasoning in language models, built on subgroup-construction problems in SL(3, Z) with cryptographic-style verifier-prover asymmetry. Each instance presents a finitely generated subgroup as a list of integer matrices and asks for an arithmetic invariant -- index, surjection-at-prime, or membership -- that the construction-time information (N, K) pins down in O(1) closed form, but that the solver, lacking that information, must derive by either Aschbacher-classification analysis or by a membership query in SL(3, Z) of unknown decidability. The benchmark therefore distinguishes models with internalized algebraic priors (Aschbacher classes, McLaughlin's theorem, Property (T), the congruence subgroup property) from models that rely on general-purpose computation. We report empirical results across five representative reasoning traces from two state-of-the-art models. The headline result: on the index variant, one model spent 152 minutes of reasoning, explicitly identified the kernel-side membership question as the bottleneck, attempted constructive verification, and abstained with "DON'T KNOW" rather than commit to its computed cokernel candidate -- demonstrating calibrated meta-cognition on the open-decidability boundary that the benchmark was designed to probe. We argue that the benchmark exposes a four-way classification of model behavior (commit-correct, commit-wrong, abstain-correct, abstain-wrong) that standard answer-key scoring conflates.

Related papers

• A cubical formalisation of topos causal models: intervention, sheaf gluing, and the intuitionistic do-calculus [5.076419064097734]
  We give the first machine-checked account of a 1-topos core, in Cubical Agda.<n>We show that interventions and Pearl's rules are stable under every topology.<n>We add a phenomenon the programme does not consider: a machine-checked contextuality obstruction.
  arXiv  Detail & Related papers  (2026-07-17T05:05:31Z)
• An Algebraic View of the Expressivity of Recurrent Language Models [50.88598486616582]
  The paper develops a unified algebraic account of the expressivity of recurrent neural networks.<n>The same architecture cannot implement an even-modulus counter once floating-point recurrences are enforced, yet realizes every even-modulus counter under unsigned-integer quantization.
  arXiv  Detail & Related papers  (2026-06-01T06:47:58Z)
• Evaluating Local Explainability Metrics for Machine Learning Models on Tabular Data [1.423280626666929]
  An explanation can appear plausible to humans but fail to capture the internal reasoning of a model.<n>This paper studies the trustworthiness of local explainability techniques when applied to complex tabular classification tasks.
  arXiv  Detail & Related papers  (2026-05-26T19:38:10Z)
• NOETHER: A Constructive Framework for Metamorphic Pattern Discovery from Operator Algebras [6.691613382732758]
  Metamorphic Testing is recognised in IEEE/ISO software-testing standards and increasingly recommended for AI systems.<n>Existing approaches share an inductive grounding that leaves three foundational questions open.<n>We propose a framework whose downstream step from operator algebra to MetaPattern set is mechanical and provable.
  arXiv  Detail & Related papers  (2026-05-17T11:18:35Z)
• Decidable By Construction: Design-Time Verification for Trustworthy AI [0.0]
  A prevailing assumption in machine learning is that model correctness must be enforced after the fact.<n>We observe that the properties determining whether an AI model is numerically stable, computationally correct, or consistent with a physical domain do not necessarily demand post hoc enforcement.
  arXiv  Detail & Related papers  (2026-03-26T13:09:36Z)
• Optimal Classification of Three-Qubit Entanglement with Cascaded Support Vector Machine [0.0]
  We introduce a systematic framework for three-qubit entanglement classification using a cascaded architecture of Support Vector Machine (SVM) classifiers.<n>We construct three distinct witness models ($mathcalM_B$, $mathcalM_W$, and $mathcalM_GHZ$) that sequentially discriminate between entanglement classes.<n>The proposed Cascaded model achieves an overall classification accuracy of $95%$ on a comprehensive dataset of mixed states.
  arXiv  Detail & Related papers  (2026-02-17T12:49:41Z)
• Unifying Tree Search Algorithm and Reward Design for LLM Reasoning: A Survey [92.71325249013535]
  Deliberative tree search is a cornerstone of Large Language Model (LLM) research.<n>This paper introduces a unified framework that deconstructs search algorithms into three core components.
  arXiv  Detail & Related papers  (2025-10-11T03:29:18Z)
• A Markov Categorical Framework for Language Modeling [9.910562011343009]
  Autoregressive language models achieve remarkable performance, yet a unified theory explaining their internal mechanisms, how training shapes their representations, and enables complex behaviors, remains elusive.<n>We introduce a new analytical framework that models the single-step generation process as a composition of information-processing stages using the language of Markov categories.<n>This work presents a powerful new lens for understanding how information flows through a model and how the training objective shapes its internal geometry.
  arXiv  Detail & Related papers  (2025-07-25T13:14:03Z)
• Learning Identifiable Structures Helps Avoid Bias in DNN-based Supervised Causal Learning [56.22841701016295]
  Supervised Causal Learning (SCL) is an emerging paradigm in this field.<n>Existing Deep Neural Network (DNN)-based methods commonly adopt the "Node-Edge approach"
  arXiv  Detail & Related papers  (2025-02-15T19:10:35Z)
• Identification and Estimation of Simultaneous Equation Models Using Higher-Order Cumulant Restrictions [5.882065571122133]
  Identifying structural parameters in linear simultaneous-equation models is a longstanding challenge.<n>We show that neither zero covariance proofs nor whitening is necessary to identify structural parameters.<n>Our framework provides a transparent overidentification test.
  arXiv  Detail & Related papers  (2025-01-12T11:27:39Z)
• Networks of Networks: Complexity Class Principles Applied to Compound AI Systems Design [63.24275274981911]
  Compound AI Systems consisting of many language model inference calls are increasingly employed. In this work, we construct systems, which we call Networks of Networks (NoNs) organized around the distinction between generating a proposed answer and verifying its correctness. We introduce a verifier-based judge NoN with K generators, an instantiation of "best-of-K" or "judge-based" compound AI systems.
  arXiv  Detail & Related papers  (2024-07-23T20:40:37Z)
• TRIGO: Benchmarking Formal Mathematical Proof Reduction for Generative Language Models [68.65075559137608]
  We propose TRIGO, an ATP benchmark that not only requires a model to reduce a trigonometric expression with step-by-step proofs but also evaluates a generative LM's reasoning ability on formulas. We gather trigonometric expressions and their reduced forms from the web, annotate the simplification process manually, and translate it into the Lean formal language system. We develop an automatic generator based on Lean-Gym to create dataset splits of varying difficulties and distributions in order to thoroughly analyze the model's generalization ability.
  arXiv  Detail & Related papers  (2023-10-16T08:42:39Z)
• Knowledge Base Question Answering by Case-based Reasoning over Subgraphs [81.22050011503933]
  We show that our model answers queries requiring complex reasoning patterns more effectively than existing KG completion algorithms. The proposed model outperforms or performs competitively with state-of-the-art models on several KBQA benchmarks.
  arXiv  Detail & Related papers  (2022-02-22T01:34:35Z)
• Prolog-based agnostic explanation module for structured pattern classification [2.0824228840987447]
  This paper presents a Prolog-based reasoning module to generate counterfactual explanations. It can also resolve what-if queries using the ground-truth labels instead of the predicted ones. Overall, our approach comprises four well-defined stages that can be applied to any structured pattern classification problem.
  arXiv  Detail & Related papers  (2021-12-23T15:28:40Z)
• GroupifyVAE: from Group-based Definition to VAE-based Unsupervised Representation Disentanglement [91.9003001845855]
  VAE-based unsupervised disentanglement can not be achieved without introducing other inductive bias. We address VAE-based unsupervised disentanglement by leveraging the constraints derived from the Group Theory based definition as the non-probabilistic inductive bias. We train 1800 models covering the most prominent VAE-based models on five datasets to verify the effectiveness of our method.
  arXiv  Detail & Related papers  (2021-02-20T09:49:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.