When Can Depth Replace Precision? A Resource Theory of Quantized Neural Computation

• URL: http://arxiv.org/abs/2607.23390v1
• Date: Sat, 25 Jul 2026 23:29:23 GMT
• Title: When Can Depth Replace Precision? A Resource Theory of Quantized Neural Computation
• Abstract summary: Pure schedules approach the relaxed class at rate $O(D-1)$ under bounded-variation time dependence.<n> Execution arithmetic can reverse this conclusion.<n>For coherent depth-$L$ first-order high-precision exponents, accuracy matching requires $D=(L)$.
• Score: 14.270378035741404
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: When can additional low-bit residual computation replace missing numerical precision for a fixed input-output map? We model a quantized residual system over a fixed horizon as a pure schedule selecting fields from a declared low-bit operation library, and use relaxed controls to characterize its infinite-depth limit. The distance from the target to the closed relaxed reachable set is the exact structural floor: no increase in depth can remove it for that library. Pure schedules approach the relaxed class at rate $O(D^{-1})$ under bounded-variation time dependence and $O(D^{-\vartheta}+D^{-1})$ under Holder dependence of exponent $\vartheta$. Execution arithmetic can reverse this conclusion: full-state write-back introduces a $Dρ_z$ penalty and can freeze residual updates, whereas increment error feedback replaces this growth by a bounded carry term and obeys an exact common-lattice conservation law. A fixed-teacher converse makes this rate sharp: for coherent depth-$L$ first-order high-precision comparators, accuracy matching requires $D=Θ(L)$. Learned codebooks add a metadata resource, while state-dependent routing introduces hybrid event conditions. Verified primal and dual bounds yield feasible, impossible, or unresolved decisions before training. Companion software implements the workflow, and Lean 4 machine-checks the exact discrete core. Depth replaces precision only relative to a declared library, horizon, execution semantics, and routing model.

Related papers

• A Single Fixed Shallow Circuit for Classical Shadows of Arbitrary n-Qubit States [2.5735476569508995]
  We construct a single fixed shallow quantum analyzer whose Born outcomes replace externally sampled settings as labels for reusable shadow snapshots.<n>The analyzer combines a freshly prepared $n$-qubit fiducial register $A$ with parallel Bell readout of $A$ and the unknown system $S$, producing one $2n$-bit record per copy.
  arXiv  Detail & Related papers  (2026-09-07T04:36:39Z)
• Constrained Online Learning with Noisy Constraint Values [55.29259818039367]
  Under common feasibility, our LEDGER algorithm attains $O(sqrt T)$ expected regret and $O(sqrtTlog(eT))$ expected budget violation.<n>Neither a Slater condition, independence between feedback channels, nor an absolute constraint-value bound is needed.
  arXiv  Detail & Related papers  (2026-09-07T01:38:41Z)
• DeepLoop: Depth Scaling for Looped Transformers [91.7922038545625]
  Looped Transformers scale sequential computation by applying a compact stack of physical blocks for multiple rounds.<n>We formalize this tied-depth effect through a first-order perturbation bound controlled by a visit-alignment coefficient.<n>The resulting method, textbfDeepLoop, keeps the Post-LN DeepNorm architecture and sets $= (2N)1/2$ and $=(8N)-1/2$ for unrolled depth.
  arXiv  Detail & Related papers  (2026-07-15T06:37:05Z)
• Compander-Aligned Query Geometry for Quantized Zeroth-Order Optimization [12.92201130466754]
  CAQ-ZO forms one-grid-step Rademacher stencils $z pm r$ in $z = (x)$, maps endpoints back through $-1$, and updates in $z$.<n>Our theory proves the grid-spanarity, decomposes endpoint-rounding estimator residuals, and gives station bounds in which generic off-grid queries retain a $2/2$ residual channel while CAQ-ZO makes the query-time residual exactly zero.
  arXiv  Detail & Related papers  (2026-05-11T14:55:20Z)
• Trust Region Masking for Long-Horizon LLM Reinforcement Learning [20.589897184824878]
  Policy gradient methods for large language models optimize a surrogate objective computed from samples of a rollout policy.<n>When $_textroll ne _$, there is approximation error between the surrogate and the true objective.<n>We propose Trust Region Masking (TRM), which excludes entire sequences from gradient computation if any token violates the trust region.
  arXiv  Detail & Related papers  (2025-12-28T20:41:59Z)
• INC: An Indirect Neural Corrector for Auto-Regressive Hybrid PDE Solvers [61.84396402100827]
  We propose the Indirect Neural Corrector ($mathrmINC$), which integrates learned corrections into the governing equations.<n>$mathrmINC$ reduces the error amplification on the order of $t-1 + L$, where $t$ is the timestep and $L$ the Lipschitz constant.<n>We test $mathrmINC$ in extensive benchmarks, covering numerous differentiable solvers, neural backbones, and test cases ranging from a 1D chaotic system to 3D turbulence.
  arXiv  Detail & Related papers  (2025-11-16T20:14:28Z)
• Catch Your Breath: Adaptive Computation for Self-Paced Sequence Production [55.76222360698305]
  We explore a class of supervised training objectives that allow a language model to dynamically and autonomously scale the number of compute steps used for each input token.<n>For any token, the model can request additional compute steps by emitting a don't know> output.<n>We find that the CYB model requests additional steps when doing so improves accuracy, and the model adapts its processing time to token-level complexity and context.
  arXiv  Detail & Related papers  (2025-10-13T21:07:05Z)
• Arithmetic-Mean $μ$P for Modern Architectures: A Unified Learning-Rate Scale for CNNs and ResNets [9.94514344279733]
  Arithmetic-Mean $mu$P constrains not each individual layer but the network-wide average one-step pre-activation second moment to a constant scale.<n>We prove that, for one- and two-dimensional convolutional networks, the maximal-update learning rate satisfies $etastar(L)propto L-3/2$; with zero padding, boundary effects are constant-level as $Ngg k$.
  arXiv  Detail & Related papers  (2025-10-05T19:22:50Z)
• The Alignment Bottleneck [0.0]
  We model the loop as a two-stage cascade $U to H to Y$ given $S$, with cognitive capacity $C_textcog|S$ and average total capacity $barC_texttot|S$.<n>It pairs a data size-independent Fano lower bound proved on a separable codebook mixture with a PAC-Bayes upper bound whose KL term is controlled by the same channel via $m, barC_texttot|S$.
  arXiv  Detail & Related papers  (2025-09-19T12:38:30Z)
• Control of the von Neumann Entropy for an Open Two-Qubit System Using Coherent and Incoherent Drives [50.24983453990065]
  This article is devoted to developing an approach for manipulating the von Neumann entropy $S(rho(t))$ of an open two-qubit system with coherent control and incoherent control inducing time-dependent decoherence rates. The following goals are considered: (a) minimizing or maximizing the final entropy $S(rho(T))$; (b) steering $S(rho(T))$ to a given target value; (c) steering $S(rho(T))$ to a target value and satisfying the pointwise state constraint $S(
  arXiv  Detail & Related papers  (2024-05-10T10:01:10Z)
• Naive Exploration is Optimal for Online LQR [49.681825576239355]
  We show that the optimal regret scales as $widetildeTheta(sqrtd_mathbfu2 d_mathbfx T)$, where $T$ is the number of time steps, $d_mathbfu$ is the dimension of the input space, and $d_mathbfx$ is the dimension of the system state. Our lower bounds rule out the possibility of a $mathrmpoly(logT)$-regret algorithm, which had been
  arXiv  Detail & Related papers  (2020-01-27T03:44:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.