Fixed-Point Reasoners: Stable and Adaptive Deep Looped Transformers

• URL: http://arxiv.org/abs/2606.18206v1
• Date: Tue, 16 Jun 2026 17:36:33 GMT
• Title: Fixed-Point Reasoners: Stable and Adaptive Deep Looped Transformers
• Abstract summary: We propose a Transformer-based Fixed-Point Reasoning Model that uses fixed-point convergence as an end-to-end halting mechanism in a looped architecture.<n>FPRM is effective on common reasoning benchmarks, namely Sudoku, Maze, state-tracking, and ARC-AGI.
• Score: 72.66457738838515
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Looped architectures provide an inductive bias toward learning step-by-step procedures for tasks that require compositional reasoning. The number of effective layers reached by looping determines the quality of the solution these models find. Like deep architectures, looped architectures are prone to a signal propagation problem induced by depth as the halting decision is postponed. In this paper, we address this signal propagation issue using pre-norm layers and residual scaling. Building on these architectural modifications, we propose FPRM, a Transformer-based Fixed-Point Reasoning Model that uses fixed-point convergence as an end-to-end halting mechanism in a looped architecture. We show that fixed-point halting allows FPRM to adapt its compute to task difficulty. FPRM is effective on common reasoning benchmarks, namely Sudoku, Maze, state-tracking, and ARC-AGI.

Related papers

• CRISP: Cliff-awaRe Input-adaptive Sparse Prefilling with Structural-Mass-Motivated Routing [90.63350360476294]
  We present CRISP (Cliff-awaRe Input-adaptive Sparse Prefilling), which identifies and addresses two structural challenges in this dynamic routing paradigm.<n>We show that the routing decision can be read directly off the structure of the proxy attention map.<n>We formalize the post-softmax mass cliff and demonstrate theoretically that strictly cumulative coverage thresholds accumulate O(n) background noise at long contexts.
  arXiv  Detail & Related papers  (2026-09-01T22:49:46Z)
• Automating Variational Quantum Sensing through Reinforcement-Learned Circuit Structures [8.108113752089812]
  textscAutoQSense is a reinforcement-learning framework that searches circuit architectures using Fisher-information-based objectives.<n>Results show that the learned architectures recover known benchmark strategies, adapt to dephasing noise, and outperform fixed hardware-efficient anstze.
  arXiv  Detail & Related papers  (2026-08-18T09:47:37Z)
• When Efficiency Becomes Fragility: Exploiting Dynamic Routing Vulnerabilities in Adaptive UAV Tracking [32.2355419863971]
  Resource constraints on UAV platforms have driven a paradigm shift in aerial tracking, from pursuing performance toward balancing accuracy with efficiency.<n> Adaptive Transformer Trackers, which leverage an input-dependent dynamic routing architecture, have emerged as a representative solution to this challenge.<n>We reveal that behind this computation-on-demand flexibility hides a critical structural flaw: the Lipschitz singularity of computational path decisions.
  arXiv  Detail & Related papers  (2026-08-04T16:29:35Z)
• Adaptive Recurrent Message Passing for Test Time Computing on Graphs [80.56829418752513]
  We show that recurrent graph models can overcome the intrinsic mismatch between graph data and architectural designs.<n>We propose AdaR, an Adaptive Recurrent graph model, empowering flexible test-time computing on various downstream tasks without changing model parameters.
  arXiv  Detail & Related papers  (2026-06-21T12:14:11Z)
• Runtime Burden Allocation for Structured LLM Routing in Agentic Expert Systems: A Full-Factorial Cross-Backend Methodology [16.053669481561354]
  As large language models (LLMs) become core control components in agentic AI systems, reliable structured routing must balance correctness, latency, and implementation cost.<n>We show that this balance is shaped not only by prompts or schemas, but also by how structural work is allocated across the generation stack.<n>We evaluate this formulation through a comprehensive full-factorial benchmark covering 48 deployment configurations and 15,552 requests across OpenAI, Gemini, and Llama backends.
  arXiv  Detail & Related papers  (2026-03-26T18:50:46Z)
• Adaptive Linear Path Model-Based Diffusion [52.84663832658799]
  We introduce Linear Path Model-Based Diffusion (LP-MBD), which replaces the variance-preserving schedule with a flow-matching linear probability path.<n>We also propose Adaptive LP-MBD (ALP-MBD), which leverages reinforcement learning to adjust diffusion steps and noise levels according to task complexity and environmental conditions.
  arXiv  Detail & Related papers  (2026-02-02T21:33:03Z)
• SpanNorm: Reconciling Training Stability and Performance in Deep Transformers [55.100133502295996]
  We propose SpanNorm, a novel technique designed to resolve the dilemma by integrating the strengths of both paradigms.<n>We provide a theoretical analysis demonstrating that SpanNorm, combined with a principled scaling strategy, maintains bounded signal variance throughout the network.<n> Empirically, SpanNorm consistently outperforms standard normalization schemes in both dense and Mixture-of-Experts (MoE) scenarios.
  arXiv  Detail & Related papers  (2026-01-30T05:21:57Z)
• Backscatter Device-aided Integrated Sensing and Communication: A Pareto Optimization Framework [59.30060797118097]
  Integrated sensing and communication (ISAC) systems potentially encounter significant performance degradation in densely obstructed urban non-line-of-sight scenarios.<n>This paper proposes a backscatter approximation (BD)-assisted ISAC system, which leverages passive BDs naturally distributed in environments of enhancement.
  arXiv  Detail & Related papers  (2025-07-12T17:11:06Z)
• FX-DARTS: Designing Topology-unconstrained Architectures with Differentiable Architecture Search and Entropy-based Super-network Shrinking [19.98065888943856]
  Strong priors are imposed on the search space of Differentiable Architecture Search (DARTS)<n>This paper aims to reduce these prior constraints by eliminating restrictions on cell topology and modifying the discretization mechanism for super-networks.<n>FX-DARTS is capable of exploring a set of neural architectures with competitive trade-offs between performance and computational complexity.
  arXiv  Detail & Related papers  (2025-04-25T08:34:29Z)
• Diffusion & Adversarial Schrödinger Bridges via Iterative Proportional Markovian Fitting [89.38017281709686]
  We show a close connection between the modified version of IMF and the Iterative Proportional Fitting (IPF) procedure.<n>We refer to this combined approach as the Iterative Proportional Markovian Fitting (IPMF) procedure.
  arXiv  Detail & Related papers  (2024-10-03T15:43:17Z)
• Decomposable Transformer Point Processes [2.1756081703276]
  We propose a framework where the advantages of the attention-based architecture are maintained and the limitation of the thinning algorithm is circumvented. The proposed method attains state-of-the-art performance in predicting the next event of a sequence given its history.
  arXiv  Detail & Related papers  (2024-09-26T13:22:58Z)
• DepGraph: Towards Any Structural Pruning [68.40343338847664]
  We study general structural pruning of arbitrary architecture like CNNs, RNNs, GNNs and Transformers. We propose a general and fully automatic method, emphDependency Graph (DepGraph), to explicitly model the dependency between layers and comprehensively group parameters for pruning. In this work, we extensively evaluate our method on several architectures and tasks, including ResNe(X)t, DenseNet, MobileNet and Vision transformer for images, GAT for graph, DGCNN for 3D point cloud, alongside LSTM for language, and demonstrate that, even with a
  arXiv  Detail & Related papers  (2023-01-30T14:02:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.