What Is the Minimum Architecture for Prolepsis? Early Irrevocable Commitment Across Tasks in Small Transformers

• URL: http://arxiv.org/abs/2604.15010v1
• Date: Thu, 16 Apr 2026 13:38:34 GMT
• Title: What Is the Minimum Architecture for Prolepsis? Early Irrevocable Commitment Across Tasks in Small Transformers
• Abstract summary: We introduce textbfprolepsis: a transformer commits early, task-specific attention heads sustain the commitment, and no layer corrects it.<n>All experiments run on a single consumer GPU (16,GB VRAM)
• Score: 0.7310043452300737
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: When do transformers commit to a decision, and what prevents them from correcting it? We introduce \textbf{prolepsis}: a transformer commits early, task-specific attention heads sustain the commitment, and no layer corrects it. Replicating \citeauthor{lindsey2025biology}'s (\citeyear{lindsey2025biology}) planning-site finding on open models (Gemma~2 2B, Llama~3.2 1B), we ask five questions. (Q1)~Planning is invisible to six residual-stream methods; CLTs are necessary. (Q2)~The planning-site spike replicates with identical geometry. (Q3)~Specific attention heads route the decision to the output, filling a gap flagged as invisible to attribution graphs. (Q4)~Search requires ${\leq}16$ layers; commitment requires more. (Q5)~Factual recall shows the same motif at a different network depth, with zero overlap between recurring planning heads and the factual top-10. Prolepsis is architectural: the template is shared, the routing substrates differ. All experiments run on a single consumer GPU (16\,GB VRAM).

Related papers

• MindTopo: Can Foundation Models Reason in Topological Space? [52.56750995613252]
  We introduce MindTopo, a benchmark of topological intuition across five properties grounded in cognitive science and formal topology.<n>MindTopo evaluates each property at two cognitive levels. Reasoning asks a model to identify topological relations or infer how they change.<n>Every MLLM performs better on reasoning than on planning, and the best-performing model remains far below observed human performance.
  arXiv  Detail & Related papers  (2026-09-10T17:54:32Z)
• What Survives When You Compress a Recursive Reasoner for the Edge? [15.460381731793975]
  Recursive reasoning models can solve complex structured tasks with only a few million parameters by repeatedly updating a latent state.<n>We find that aggressive compression preserves local prediction but destroys global reasoning.<n>We introduce carry-trajectory fidelity, similarity to the full-precision reasoning path, as a label-free signal that predicts this damage and its recovery.
  arXiv  Detail & Related papers  (2026-06-25T00:47:15Z)
• Task Structure Reverses Layerwise State Encoding in Sequence Models [8.403971471573607]
  We find that the same architecture reverses this profile when the task changes.<n>In fine-tuned Mamba-130M and Pythia-160M, Parity is concentrated late in Mamba and the recurrent baselines and built gradually by Transformer.<n>The same flip appears in fine-tuned Mamba-130M and Pythia-160M, and the Pythia Dyck bottleneck persists at 410M.
  arXiv  Detail & Related papers  (2026-05-30T23:26:29Z)
• Diagnosing Overhead in Dispatch Operations: Cross-architecture Observatory [1.081571058570587]
  AlltoAll dispatch is the dominant bottleneck of MoE expert parallelism.<n>We introduce DODOCO to test two assumptions about the workload.<n> scaling EP changes the per-expert max/mean token ratio by at most 5% within every architecture's measurable range.
  arXiv  Detail & Related papers  (2026-05-20T10:14:00Z)
• ARGOS: Who, Where, and When in Agentic Multi-Camera Person Search [49.61286310968402]
  We introduce ARGOS, the first benchmark and framework that reformulates multi-camera person search as an interactive reasoning problem.<n>An ARGOS agent receives a vague witness statement and must decide what to ask, when to invoke spatial or temporal tools, and how to interpret ambiguous responses.<n>The benchmark comprises 2,691 tasks across 14 real-world scenarios in three progressive tracks: semantic perception (Who), spatial reasoning (Where), and temporal reasoning (When)
  arXiv  Detail & Related papers  (2026-04-14T14:06:19Z)
• The Spectral Geometry of Thought: Phase Transitions, Instruction Reversal, Token-Level Dynamics, and Perfect Correctness Prediction in How Transformers Reason [4.28787537081191]
  We find that large language models exhibit emphspectral phase transitions in their hidden activation spaces when engaging in reasoning versus factual recall.<n>We establish a comprehensive emphspectral theory of reasoning in transformers, revealing that the geometry of thought is universal in direction, architecture-specific, and predictive of outcome.
  arXiv  Detail & Related papers  (2026-04-03T09:18:57Z)
• GraphWalker: Agentic Knowledge Graph Question Answering via Synthetic Trajectory Curriculum [22.56991897386462]
  Agentic knowledge graph question answering (KGQA) requires an agent to iteratively interact with knowledge graphs (KGs)<n>Existing approaches often restrict agent exploration, while current training pipelines usually confine reasoning to predefined trajectories.<n>This paper proposes textitGraphWalker,' a novel agentic KGQA framework that addresses these challenges through textitAutomated Trajectory Synthesis and textitStage-wise Fine-tuning
  arXiv  Detail & Related papers  (2026-03-30T14:56:59Z)
• Tabula RASA: Exposing and Breaking the Relational Bottleneck in Transformers [0.0]
  RASA (Relation-Aware Sparse Attention) is a minimal architectural modification that provides structural inductive bias for relational reasoning.<n>Our results demonstrate that minimal architectural modifications, grounded in complexity-theoretic analysis, can substantially improve multi-hop reasoning.
  arXiv  Detail & Related papers  (2026-02-02T21:35:39Z)
• ChartVerse: Scaling Chart Reasoning via Reliable Programmatic Synthesis from Scratch [57.01439313241121]
  We introduce Rollout Posterior Entropy (RPE), a novel metric that quantifies chart complexity.<n>We also develop truth-anchored inverse QA synthesis to guarantee reasoning rigor.<n>To further elevate difficulty and reasoning depth, we filter samples based on model fail-rate and distill high-quality Chain-of-Thought (CoT) reasoning.
  arXiv  Detail & Related papers  (2026-01-20T05:11:44Z)
• Automated Circuit Interpretation via Probe Prompting [0.0]
  We present probe prompting, an automated pipeline that transforms attribution graphs into compact, interpretable subgraphs.<n>Across five prompts, probe-prompted subgraphs preserve high explanatory coverage while compressing complexity.<n>Compared to geometric clustering baselines, concept-aligned groups exhibit higher behavioral coherence.
  arXiv  Detail & Related papers  (2025-11-10T11:53:36Z)
• Exploring Diffusion Transformer Designs via Grafting [82.91123758506876]
  We present grafting, a simple approach for editing pretrained diffusion transformers (DiTs) to materialize new architectures under small compute budgets.<n>We show that new diffusion model designs can be explored by grafting pretrained DiTs, with edits ranging from operator replacement to architecture restructuring.
  arXiv  Detail & Related papers  (2025-06-05T17:59:40Z)
• Attention Map Guided Transformer Pruning for Edge Device [98.42178656762114]
  Vision transformer (ViT) has achieved promising success in both holistic and occluded person re-identification (Re-ID) tasks. We propose a novel attention map guided (AMG) transformer pruning method, which removes both redundant tokens and heads. Comprehensive experiments on Occluded DukeMTMC and Market-1501 demonstrate the effectiveness of our proposals.
  arXiv  Detail & Related papers  (2023-04-04T01:51:53Z)
• Graph Attention with Hierarchies for Multi-hop Question Answering [19.398300844233837]
  We present two extensions to the SOTA Graph Neural Network (GNN) based model for HotpotQA. Experiments on HotpotQA demonstrate the efficiency of the proposed modifications.
  arXiv  Detail & Related papers  (2023-01-27T15:49:50Z)
• Conditional Generation with a Question-Answering Blueprint [84.95981645040281]
  We advocate planning as a useful intermediate representation for rendering conditional generation less opaque and more grounded. We obtain blueprints automatically by exploiting state-of-the-art question generation technology. We develop Transformer-based models, each varying in how they incorporate the blueprint in the generated output.
  arXiv  Detail & Related papers  (2022-07-01T13:10:19Z)
• Decoupled Multi-task Learning with Cyclical Self-Regulation for Face Parsing [71.19528222206088]
  We propose a novel Decoupled Multi-task Learning with Cyclical Self-Regulation for face parsing. Specifically, DML-CSR designs a multi-task model which comprises face parsing, binary edge, and category edge detection. Our method achieves the new state-of-the-art performance on the Helen, CelebA-HQ, and LapaMask datasets.
  arXiv  Detail & Related papers  (2022-03-28T02:12:30Z)
• An Information Theory-inspired Strategy for Automatic Network Pruning [97.03772272417599]
  Deep convolution neural networks are well known to be compressed on devices with resource constraints.<n>Most existing network pruning methods require laborious human efforts and prohibitive computation resources.<n>We propose an information theory-inspired strategy for automatic model compression.
  arXiv  Detail & Related papers  (2021-08-19T07:03:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.