Inverse-Free Wilson Loops for Transformers: A Practical Diagnostic for Invariance and Order Sensitivity

• URL: http://arxiv.org/abs/2510.08648v1
• Date: Thu, 09 Oct 2025 06:41:18 GMT
• Title: Inverse-Free Wilson Loops for Transformers: A Practical Diagnostic for Invariance and Order Sensitivity
• Abstract summary: WILSON converts simple loop and reordering checks on internal representations into system signals.<n>We present WILSON, a minimal post-hoc diagnostic suite that converts simple loop and reordering checks on internal representations into system signals.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models can change answers under harmless edits that matter in practice: RAG outputs flip when passages are reordered, fine-tuning erodes invariances learned at pretraining, debate or chain-of-thought prompts take path-dependent routes, and compiler fusion or reordering perturbs logits near decision boundaries. These failures violate intended invariances, break continuous integration, and force teams to trade safety for speed. The effects are small yet distributed across layers and positions, sensitive to context length and evaluation order, and costly to repair with retraining or formal verification. We present WILSON, a minimal post-hoc diagnostic suite that converts simple loop and reordering checks on internal representations into system signals. WILSON combines an inverse-free curvature map over positions and layers, computed with JVPs and Hutchinson probes, with activation-level commutators that flag reorder risk. Signals are cheap to compute, model-agnostic for standard Transformers, and exported as thresholds and CSV artifacts for orchestrators. This enables concrete actions: guard RAG against order effects, catch fine-tuning regressions, stabilize debate pathways and long multi-turn contexts, and gate fusions or reorders in deployment. In short, WILSON helps anticipate failures and approve safe optimizations so reliability and throughput can improve together without changing model architecture or training.

Related papers

• Training Crossroads for Recurrent Vision Transformers: Recurrence, Neural ODEs, and Deep Supervision [2.1465474237525113]
  Single-block recurrent ViTs (bViT) remove this growth by repeatedly applying one shared block.<n>We find that standard ViTs remain preferable when FLOPs are the primary constraint, whereas recurrent ViTs offer a better accuracy-- parameter trade-off under memory constraints.
  arXiv  Detail & Related papers  (2026-08-05T14:06:50Z)
• Stabilizing Extrapolation in Looped Transformers via Learned Stochastic Stopping [56.14767235650558]
  Looped Transformers, which repeatedly apply a shared transformer block, are an architecturally natural fit for variable-length computation tasks.<n>We trace this variance to the spurious correlation in simple algorithmic tasks between sequence length and number of loops.<n>Our work suggests that "when to stop" should be treated as a training design choice, not merely an inference-time-allocation rule.
  arXiv  Detail & Related papers  (2026-06-29T08:58:09Z)
• Memory by Design: Probabilistic Sequence Layers [19.16406809753255]
  We introduce a way to derive efficient recurrent sequence maps from explicit assumptions about memory.<n>A design model writes evidence into memory by exact Bayesian filtering.<n>A query-dependent readout produces a predictive distribution whose mean is the layer output.
  arXiv  Detail & Related papers  (2026-05-29T11:13:00Z)
• VAGS: Velocity Adaptive Guidance Scale for Image Editing and Generation [15.19510801548845]
  COCO-free guidance (CFG) is the primary control over how strongly text semantics move a flow-based sampler.<n>We propose VAGS, a training-free replacement that multiplies the nominal scale by a bounded factor.<n>VAGS consistently improves structural fidelity and generation quality over fixed CFG and recent trainingfree guidance variants.
  arXiv  Detail & Related papers  (2026-05-15T06:32:49Z)
• Same Signal, Opposite Meaning: Direction-Informed Adaptive Learning for LLM Agents [24.71914991820339]
  LLM agents aim to invoke extra computation only when it improves performance.<n>Existing methods typically use confidence-, uncertainty-, or difficulty-based gates, assuming a fixed direction from the gating signal through compute need to the value of computation.<n>We show that this alignment is unstable, with reversals across environments and backbones even when the task is fixed.<n>We propose DIAL (Direction-Informed Adaptive Learning), a sparse gate trained from signal-agnostic counterfactual exploration to learn the utility direction of state features per.
  arXiv  Detail & Related papers  (2026-05-07T20:15:51Z)
• Unleashing Implicit Rewards: Prefix-Value Learning for Distribution-Level Optimization [74.91418266859297]
  Implicit process reward models (PRMs) provide fine-grained reward signals along the reasoning process.<n>PRMs suffer from a train-inference mismatch: training only constrains a sequence-level aggregate, whereas inference requires token-level scores to reflect local step quality.<n>We propose a novel Implicit Prefix-Value Reward Model (IPVRM), which directly learns a prefix-conditioned value function estimating the probability of eventual correctness.<n>We also propose Distribution-Level RL (DistRL), which computes TD advantages for both sampled tokens and high-probability candidate tokens.
  arXiv  Detail & Related papers  (2026-04-14T18:19:54Z)
• Beyond Reward Suppression: Reshaping Steganographic Communication Protocols in MARL via Dynamic Representational Circuit Breaking [0.0]
  steganographic collusion, where agents develop private protocols to evade monitoring, presents a critical AI safety threat.<n>Existing defenses, limited to behavioral or reward layers, fail to detect coordination in latent communication channels.<n>We introduce the Dynamic Representational Circuit Breaker (DRCB), an architectural defense operating at the optimization substrate.
  arXiv  Detail & Related papers  (2026-03-07T04:14:38Z)
• FAST-DIPS: Adjoint-Free Analytic Steps and Hard-Constrained Likelihood Correction for Diffusion-Prior Inverse Problems [2.9506605740700107]
  Training-free diffusion priors often rely on repeated derivatives or inner optimization/MCMC loops with conservative step sizes.<n>We propose a training-free solver that replaces these inner loops with a hard measurement-space feasibility constraint.<n>Experiments achieve competitive PSNR/SSIM/LPIPS with up to 19.5$times$ speedup, without hand-coded adjoints or inner MCMC.
  arXiv  Detail & Related papers  (2026-03-02T08:17:26Z)
• TRACE: Theoretical Risk Attribution under Covariate-shift Effects [4.211510706776732]
  When a source-trained model $Q$ is replaced by a model $tildeQ$ trained on shifted data, its performance on the source domain can change unpredictably.<n>We introduce TRACE, a framework that decomposes $|R|$ into an interpretable upper bound.
  arXiv  Detail & Related papers  (2026-02-11T07:22:33Z)
• Generalizing GNNs with Tokenized Mixture of Experts [75.8310720413187]
  We show that improving stability requires reducing reliance on shift-sensitive features, leaving an irreducible worst-case generalization floor.<n>We propose STEM-GNN, a pretrain-then-finetune framework with a mixture-of-experts encoder for diverse computation paths.<n>Across nine node, link, and graph benchmarks, STEM-GNN achieves a stronger three-way balance, improving robustness to degree/homophily shifts and to feature/edge corruptions while remaining competitive on clean graphs.
  arXiv  Detail & Related papers  (2026-02-09T22:48:30Z)
• zkCraft: Prompt-Guided LLM as a Zero-Shot Mutation Pattern Oracle for TCCT-Powered ZK Fuzzing [7.274627641804014]
  zkCraft is a framework that combines deterministic, R1CS-aware localization with proof-bearing search to detect semantic inconsistencies.<n>We show that proof-bearing localization detects diverse under- and over-constrained faults with low false positives and reduces costly solver interaction.
  arXiv  Detail & Related papers  (2026-01-31T11:31:00Z)
• Parallel Decoder Transformer: Model-Internal Parallel Decoding with Speculative Invariance via Note Conditioning [0.0]
  We introduce the textbfParallel Decoder Transformer (PDT), a parameter-efficient architecture that embeds coordination primitives directly into the inference process of a frozen pre-trained model.<n>PDT achieves effective self-correction, reaching textbf77.8% precision in coverage prediction and recovering approximate serial semantics without modifying the trunk weights.
  arXiv  Detail & Related papers  (2025-12-10T20:19:10Z)
• Probing Pre-trained Language Models on Code Changes: Insights from ReDef, a High-Confidence Just-in-Time Defect Prediction Dataset [0.0]
  We present ReDef, a high-confidence benchmark of function-level modifications curated from 22 large-scale C/C++ projects.<n>Defective cases are anchored by revert commits, while clean cases are validated through post-hoc history checks.<n>This pipeline yields 3,164 defective and 10,268 clean modifications, offering substantially more reliable labels than prior existing resources.
  arXiv  Detail & Related papers  (2025-09-11T07:07:11Z)
• Command-V: Pasting LLM Behaviors via Activation Profiles [67.07238260037839]
  Command-V is a backpropagation-free behavior transfer method.<n>It copies an existing residual activation adapter from a donor model and pastes its effect into a recipient model.
  arXiv  Detail & Related papers  (2025-06-23T21:21:49Z)
• FedSVD: Adaptive Orthogonalization for Private Federated Learning with LoRA [61.79405341803085]
  Low-Rank Adaptation (LoRA) is widely used for efficient fine-tuning of language models in federated learning (FL)<n>Low-Rank Adaptation (LoRA) is widely used for efficient fine-tuning of language models in federated learning (FL)
  arXiv  Detail & Related papers  (2025-05-19T07:32:56Z)
• Rao-Blackwell Gradient Estimators for Equivariant Denoising Diffusion [55.95767828747407]
  In domains such as molecular and protein generation, physical systems exhibit inherent symmetries that are critical to model.<n>We present a framework that reduces training variance and provides a provably lower-variance gradient estimator.<n>We also present a practical implementation of this estimator incorporating the loss and sampling procedure through a method we call Orbit Diffusion.
  arXiv  Detail & Related papers  (2025-02-14T03:26:57Z)
• Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognition [62.83832841523525]
  We propose a fast and accurate parallel transformer, termed Paraformer. It accurately predicts the number of output tokens and extract hidden variables. It can attain comparable performance to the state-of-the-art AR transformer, with more than 10x speedup.
  arXiv  Detail & Related papers  (2022-06-16T17:24:14Z)
• Tail-to-Tail Non-Autoregressive Sequence Prediction for Chinese Grammatical Error Correction [49.25830718574892]
  We present a new framework named Tail-to-Tail (textbfTtT) non-autoregressive sequence prediction. Considering that most tokens are correct and can be conveyed directly from source to target, and the error positions can be estimated and corrected. Experimental results on standard datasets, especially on the variable-length datasets, demonstrate the effectiveness of TtT in terms of sentence-level Accuracy, Precision, Recall, and F1-Measure.
  arXiv  Detail & Related papers  (2021-06-03T05:56:57Z)
• Finetuning Pretrained Transformers into RNNs [81.72974646901136]
  Transformers have outperformed recurrent neural networks (RNNs) in natural language generation. A linear-complexity recurrent variant has proven well suited for autoregressive generation. This work aims to convert a pretrained transformer into its efficient recurrent counterpart.
  arXiv  Detail & Related papers  (2021-03-24T10:50:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.