Power law graph attention: exact generalization of scaled dot-product attention, empirical collapse at inference

• URL: http://arxiv.org/abs/2608.10288v1
• Date: Mon, 10 Aug 2026 22:45:22 GMT
• Title: Power law graph attention: exact generalization of scaled dot-product attention, empirical collapse at inference
• Abstract summary: Large Language Model from Power Law Decoder Representations (PLDR-LLM) and its attention, Power Law Graph Attention (PLGA)<n>PLGA replaces the fixed bilinear form of scaled dotproduct attention (SDPA) with a learned, input-generated bilinear operator $G_LM$.<n>PLGA contains SDPA exactly at $G_LM=I$; $A_LM$ and $A_P$ are strictly entrywise positive, with Perron-Frobenius structure on $A_LM$.
• Score: 0.5076419064097734
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The Large Language Model from Power Law Decoder Representations (PLDR-LLM) and its attention, Power Law Graph Attention (PLGA), replace the fixed bilinear form of scaled dot-product attention (SDPA) with a learned, input-generated bilinear operator $G_{LM}$, built from a positive tensor $A_{LM}$ by elementwise power laws. The architecture is fully specified, verified against pinned reference releases; claims are labeled theorem, conditional theorem, measurement, or conjecture. Unconditionally: PLGA contains SDPA exactly at $G_{LM}=I$; $A_{LM}$ and $A_P$ are strictly entrywise positive, with Perron-Frobenius structure on $A_{LM}$; the DAG regularizer has the NOTEARS walk-counting form and positivity obstructs exact acyclicity; and, under nonresonance (satisfied by standard rotary frequencies), a commutant criterion identifies which operators preserve relative-position dependence. An inference-collapse theorem: exact input invariance of deductive outputs collapses inference to generalized SDPA with a constant operator. Measured invariance: relative fluctuations of $10^{-6}$ and below; perturbation bounds quantify but do not certify cached inference; the assembled proxy misses the decoding margin. A conditional three-stage mechanism (rotary twirl, concentration, row-map contraction) is measured on a released checkpoint. Blockwise training and scoring under the global Gram are stated with explicit target exposure; on tested samples, block and sequential scoring select identical answers and agree on the published TruthfulQA probability-mass metric within $5\times 10^{-5}$ per item. Self-organized criticality enters as a phenomenological framework with an intrinsic order parameter; open claims become falsifiable conjectures. Selected proof cores are machine-checked in Lean 4.

Related papers

• Observation Design for Certified Control Authority: Projection--Estimability Separation and Active-Face Equivalence [2.7954473524479275]
  A gate executes only actions it can certify, and certifies only what its observations support.<n>This paper asks how observations should be designed to maximize the set of actions that can be safely admitted.
  arXiv  Detail & Related papers  (2026-09-05T07:57:04Z)
• Intrinsic Structure: Spectral Identifiability for Mechanistic Interpretability [51.56484100374058]
  We prove the first identifiability theorem for a mechanistic-interpretability primitive.<n>The spectrum is an identifiable, model-intrinsic fingerprint with a stated error bar, not a legible decomposition.
  arXiv  Detail & Related papers  (2026-08-10T19:42:01Z)
• Adaptive operator-generated subspaces for effective many-body Hamiltonians [0.0]
  We present the Adaptive Clifford-Algebra Subspacesolver (A-CASE)<n>Overlap, Hamiltonian, observable, and response matrices are reconstructed from one shared Pauli-expectation bank.<n>A grouped iteration propagates finite-shot variability through thresholding, root matching, spectral weights, susceptibility, and broadening.
  arXiv  Detail & Related papers  (2026-08-01T09:51:25Z)
• When Rates Are Geometric: Rate-Certificate Transfer for Contact Splittings in Optimization [0.0]
  A contact Hamiltonian $H$ on $J1(mathbbRn)$ obeys the intrinsic decay identity $dot H = -H,partial_s H$.<n>An augmented energy $mathcalE$ built from $H$ is a continuous-time rate certificate whenever $mathcalE$ controls the objective gap.
  arXiv  Detail & Related papers  (2026-07-26T13:12:58Z)
• The SIGReg Objective as Variational Free Energy: A Theoretical Active-Inference Account of JEPA World Models [47.471225029744424]
  Joint-Embedding Predictive Architectures (JEPAs) are the dominant design for latent world models.<n>We show that the choice of anti-collapse regulariser determines whether a JEPA's training objective, a prediction loss plus a weighted embedding regulariser, is a valid Active Inference (AIF) variational free energy.
  arXiv  Detail & Related papers  (2026-07-15T08:59:36Z)
• Quasilinear Equivalence Checking for Detector Error Models [0.0]
  A Detector Error Model (DEM) is a structured representation of error mechanisms in quantum circuits.<n>We develop an equational theory for DEMs, with its associated categorical semantics.<n>We provide the first static decision procedure for DEM equivalence, with rigorous correctness guarantees.
  arXiv  Detail & Related papers  (2026-06-12T17:42:16Z)
• Imbuing Large Language Models with Bidirectional Logic for Robust Chain Repair [44.80087038178069]
  We introduce Teleological Reasoning Infilling (TRI), a training framework that endows decoder-only transformers with a native emphgoal-conditioned bridging capability.<n>At inference, TRI operates as a surgical repair module within a dual-system loop.<n>Experiments on three benchmarks demonstrate that TRI achieves state-of-the-art performance across all tasks, while reducing per-problem token expenditure by 31.2%.
  arXiv  Detail & Related papers  (2026-06-03T15:58:48Z)
• When to Trust Confidence Thresholding: Calibration Diagnostics for Pseudo-Labelled Regression [1.14219428942199]
  A standard practice is to threshold the calibrated score at a confidence cutoff and treat the hard label as the truth.<n>We develop a calibration-aware diagnostic apparatus for pseudo-labelling pipelines.<n>We derive a closed-form expression for the attenuation bias that confidence thresholding induces in the downstream regression coefficient.
  arXiv  Detail & Related papers  (2026-05-12T21:49:11Z)
• The Extrapolation Cliff in On-Policy Distillation of Near-Deterministic Structured Outputs [52.709361620508595]
  ListOPD brings a student to in-domain with an 8B-SFT baseline at one-fifth the parameters.<n>On Amazon Fashion, three pre-registered tests--a fine-grid cliff interval test, and a small-clip cross-prediction--fall within their locked prediction windows--with the small-clip value matching the closed-form prediction below grid resolution.
  arXiv  Detail & Related papers  (2026-05-09T06:48:00Z)
• Observable-Conditioned Backaction in Dynamic Circuits: A Higher-Order Context-Conditioned Kernel for Local Dynamics [0.0]
  Mid-circuit measurements are essential primitives for dynamic circuits and quantum error correction.<n> Device-level benchmarking often compresses this disturbance into low-order proxy metrics.<n>We argue that these proxies can be operationally incomplete for multiscale dynamic circuits.
  arXiv  Detail & Related papers  (2026-03-19T00:59:11Z)
• Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers [90.50039419576807]
  Reinforcement Learning with Verifiable Rewards (RLVR) trains policies against automated verifiers to avoid costly human labeling.<n>To reduce vulnerability to verifier hacking, many RLVR systems collapse rewards to binary $0,1$ during training.<n>This choice carries a cost: it introduces textitfalse negatives (rejecting correct answers, FNs) and textitfalse positives (accepting incorrect ones, FPs)
  arXiv  Detail & Related papers  (2025-10-01T13:56:44Z)
• Unsupervised Conformal Inference: Bootstrapping and Alignment to Control LLM Uncertainty [49.19257648205146]
  We propose an unsupervised conformal inference framework for generation.<n>Our gates achieve close-to-nominal coverage and provide tighter, more stable thresholds than split UCP.<n>The result is a label-free, API-compatible gate for test-time filtering.
  arXiv  Detail & Related papers  (2025-09-26T23:40:47Z)
• Almost Tight L0-norm Certified Robustness of Top-k Predictions against Adversarial Perturbations [78.23408201652984]
  Top-k predictions are used in many real-world applications such as machine learning as a service, recommender systems, and web searches. Our work is based on randomized smoothing, which builds a provably robust classifier via randomizing an input. For instance, our method can build a classifier that achieves a certified top-3 accuracy of 69.2% on ImageNet when an attacker can arbitrarily perturb 5 pixels of a testing image.
  arXiv  Detail & Related papers  (2020-11-15T21:34:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.