Exact ReLU realization of tensor-product refinement iterates

• URL: http://arxiv.org/abs/2605.03917v1
• Date: Tue, 05 May 2026 16:12:32 GMT
• Title: Exact ReLU realization of tensor-product refinement iterates
• Abstract summary: We prove that for every compactly supported continuous piecewise linear seed g:R2->R, the iterates Vn g admit exact ReLU realizations of fixed width and depth O(n)<n>This gives a first genuinely two-dimensional extension of the exact realization theory for refinement cascades.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study scalar dyadic refinement operators on R^2 of the form (Vf)(x,y) = sum_{(j,k) in Z^2} c_{j,k} f(2x-j, 2y-k), where only finitely many mask coefficients c_{j,k} are nonzero. Under a fixed support-window hypothesis, we prove that for every compactly supported continuous piecewise linear seed g:R^2->R, the iterates V^n g admit exact ReLU realizations of fixed width and depth O(n). This gives a first genuinely two-dimensional extension of the exact realization theory for refinement cascades. Using the one-dimensional exact loop-controller framework, the proof transports the tensor-product residual dynamics exactly on the product of two polygonal loops and reduces the remaining seam ambiguity to a final readout and selector step. The matrix cascade is then handled by a fixed-depth recursive block, and general compactly supported continuous piecewise linear seeds are reduced to a finite decomposition together with exact clamped gluing on the support window. This identifies the tensor-product dyadic case as a natural first multivariate instance of the loop-controller method for refinement iterates.

Related papers

• A Constant-Competitive Algorithm for Dynamic Mixture-of-Experts Serving [0.0]
  Huang, Lou, and Xiao introduced Dynamic Mixture-of-Experts Serving and gave an O(sqrt(log k))-competitive randomized algorithm for its integral primal problem.<n>We prove that the randomized primal competitive ratio is in fact Theta(1) for arbitrary numbers of experts.
  arXiv  Detail & Related papers  (2026-08-15T03:51:27Z)
• Exact ReLU realization of affine one-dimensional refinement iterates via residual memory and offset frames [0.0]
  We study vector-valued affine refinement operators of the form [ (W)(t)=sum_jinmathbbZ A_j(Mt-j)+B(t), ] with finitely supported matrix mask.<n>For (Mge 3), every finite affine iterate admits an exact fixed-width ReLU realization whose depth is (O(n)).<n>We also prove a stage-dependent extension for forcing terms in a fixed finite-dimensional continuous piecewise linear span.
  arXiv  Detail & Related papers  (2026-07-22T13:51:38Z)
• Neural network realization of binary refinement iterates via a two-chart atlas selector [0.0]
  We show that every compactly supported continuous piecewise linear seed has finite refinement iterates that admit exact ReLU realizations of fixed width and depth growing linearly.<n>The construction also gives exact readout of every continuous piecewise linear circle function satisfying the natural endpoint compatibility condition.
  arXiv  Detail & Related papers  (2026-07-21T14:11:16Z)
• Maximal Classicalization of Finite-Group Quantum Reference-Frame Noise [0.0]
  A finite quantum reference token with group-valued misalignment induces a random-unitary channel.<n>We prove that the following conditions are equivalent for a unitary representation U of a finite group G.
  arXiv  Detail & Related papers  (2026-07-14T17:07:57Z)
• Dimension Reduction for Curves: Simplified and Generalized [56.23281156053955]
  We revisit random projections for reducing the dimension of high-dimensional polygonal curves.<n>We give a simplified proof of the known $O(varepsilon-2log(nm))$ bound on the target dimension of a random projection.<n>Our proof is based on the concept of sparse oblivious subspace embeddings.
  arXiv  Detail & Related papers  (2026-07-03T08:48:35Z)
• Exact Loop Controllers for ReLU Realization of Homogeneous Curve Refinements [0.0]
  We study homogeneous refinement operators ((V)(t)=sum_jinmathbb ZA_j(Mt-j)), acting on compactly supported continuous piecewise linear curves (:mathbb Rtomathbb Rp), where (Mge2) and only finitely many matrices (A_jinmathbb Rptimes p) are nonzero.<n>We prove that the iterates (Vn) admit exact Re
  arXiv  Detail & Related papers  (2026-05-03T00:44:46Z)
• Compact Lifted Relaxations for Low-Rank Optimization [2.0052993723676895]
  We develop tractable convex relaxations for rank-constrained quadratic optimization problems over $n times m$ matrices.<n>We derive lifted semidefinite relaxations that do not require such spectral terms.<n>Overall, we obtain scalable semidefinite bounds for a broad class of low-rank quadratic problems.
  arXiv  Detail & Related papers  (2026-03-05T17:05:18Z)
• Regularized Online RLHF with Generalized Bilinear Preferences [68.44113000390544]
  We consider the problem of contextual online RLHF with general preferences.<n>We adopt the Generalized Bilinear Preference Model to capture preferences via low-rank, skew-symmetric matrices.<n>We prove that the dual gap of the greedy policy is bounded by the square of the estimation error.
  arXiv  Detail & Related papers  (2026-02-26T15:27:53Z)
• Structure-Preserving Nonlinear Sufficient Dimension Reduction for Tensors [8.679002268074683]
  We introduce two nonlinear sufficient dimension reduction methods for regressions with tensor-valued predictors.<n>Our goal is two-fold: the first is to preserve the tensor structure when performing dimension reduction, particularly the meaning of the tensor modes, for improved interpretation.<n>The second is to substantially reduce the number of parameters in dimension reduction, thereby achieving model parsimony and enhancing estimation accuracy.
  arXiv  Detail & Related papers  (2025-12-23T05:19:16Z)
• Low-Rank Tensor Recovery via Variational Schatten-p Quasi-Norm and Jacobian Regularization [49.85875869048434]
  We propose a CP-based low-rank tensor function parameterized by neural networks for implicit neural representation.<n>To achieve sparser CP decomposition, we introduce a variational Schatten-p quasi-norm to prune redundant rank-1 components.<n>For smoothness, we propose a regularization term based on the spectral norm of the Jacobian and Hutchinson's trace estimator.
  arXiv  Detail & Related papers  (2025-06-27T11:23:10Z)
• Riemannian Optimization on Relaxed Indicator Matrix Manifold [83.13494760649874]
  The indicator matrix plays an important role in machine learning, but optimizing it is an NP-hard problem.<n>We propose a new relaxation of the indicator matrix and prove that this relaxation forms a manifold, which we call the Relaxed Indicator Matrix Manifold (RIM manifold)<n>We provide several methods of Retraction, including a fast Retraction method to obtain geodesics.
  arXiv  Detail & Related papers  (2025-03-26T12:45:52Z)
• Tensor Decomposition Meets RKHS: Efficient Algorithms for Smooth and Misaligned Data [10.005398712684244]
  We decompose a multidimensional data array into a sum of outer products of finite-dimensional vectors. We refer to tensors with some infinite-dimensional modes as quasitensors. The approach of decomposing a tensor with some continuous RKHS modes is referred to as CP-HiFi.
  arXiv  Detail & Related papers  (2024-08-11T02:58:28Z)
• Interpolating between R\'enyi entanglement entropies for arbitrary bipartitions via operator geometric means [0.0]
  We introduce a new construction of subadditive and submultiplicative monotones in terms of a regularized R'enyi divergence. We show that they can be combined in a nontrivial way using weighted operator geometric means. In addition, we find lower bounds on the new functionals that are superadditive and supermultiplicative.
  arXiv  Detail & Related papers  (2022-08-30T17:56:53Z)
• Lifting the Convex Conjugate in Lagrangian Relaxations: A Tractable Approach for Continuous Markov Random Fields [53.31927549039624]
  We show that a piecewise discretization preserves better contrast from existing discretization problems. We apply this theory to the problem of matching two images.
  arXiv  Detail & Related papers  (2021-07-13T12:31:06Z)
• Competitive Mirror Descent [67.31015611281225]
  Constrained competitive optimization involves multiple agents trying to minimize conflicting objectives, subject to constraints. We propose competitive mirror descent (CMD): a general method for solving such problems based on first order information. As a special case we obtain a novel competitive multiplicative weights algorithm for problems on the positive cone.
  arXiv  Detail & Related papers  (2020-06-17T22:11:35Z)
• On Linear Stochastic Approximation: Fine-grained Polyak-Ruppert and Non-Asymptotic Concentration [115.1954841020189]
  We study the inequality and non-asymptotic properties of approximation procedures with Polyak-Ruppert averaging. We prove a central limit theorem (CLT) for the averaged iterates with fixed step size and number of iterations going to infinity.
  arXiv  Detail & Related papers  (2020-04-09T17:54:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.