Coupled Scaling: A Representational Accessibility Framework for Neural Scaling Laws

• URL: http://arxiv.org/abs/2609.03533v1
• Date: Thu, 03 Sep 2026 08:32:46 GMT
• Title: Coupled Scaling: A Representational Accessibility Framework for Neural Scaling Laws
• Abstract summary: We introduce Coupled Scaling, a framework in which finite-budget scaling depends on the relation between task structure and the geometry accessible to an architecture-optimization system.<n>A fixed- kernel derives the training-time exponent from the near-zero tail of a task-weighted spectral measure defined independently of the loss fit.
• Score: 2.345146665577353
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing theories derive neural scaling from data geometry or a specified data-model spectrum, but systems trained on the same data can scale differently when architecture or optimization changes the representations they can efficiently reach. We introduce Coupled Scaling, a task-conditioned framework in which finite-budget scaling depends on the relation between task structure and the geometry accessible to an architecture-optimization system. In a solvable mode-truncation model, loss separates into target energy outside architectural support and an unresolved supported tail. For an arbitrary priority order, the residual lies between the best-N supported tail and the tail beyond the largest completed high-value prefix. If the cumulative-tail and coverage log-rates are $γ_{A,T}$ and $ρ_{A,O,T}$, the residual exponent lies in $[ρ_{A,O,T}γ_{A,T},γ_{A,T}]$. Under bounded off-prefix gain, the completed prefix is rate-determining and $α_{A,O,T}=ρ_{A,O,T}γ_{A,T}$; for $a_{A,T,j}\asymp j^{-b_{A,T}}$, this gives $α_{A,O,T}=ρ_{A,O,T}(b_{A,T}-1)$. A fixed-kernel specialization derives the training-time exponent from the near-zero tail of a task-weighted spectral measure defined independently of the loss fit. The framework separates architectural support from finite-budget acquisition and motivates two tests: static task-relevant geometry should track loss at a common budget, while multiscale geometry should track coupling-specific exponent ordering, including reversal across contrasting tasks. An audit of released emergence trajectories identifies the controls needed for a direct factorial test that measures geometry separately from the scaling fit.

Related papers

• Mesh Graph Neural Network Framework for Accelerating Finite Element Simulation for Arbitrary Geometries [0.126928839393823]
  This work presents a mesh graph network (MGN) for predicting von Mises stress fields in 2D structural components with arbitrary hole geometries.<n>Unlike traditional machine learning approaches that use absolute node coordinates as features, the proposed model builds on existing MGN frameworks.<n>The model achieves $R2 geq 0.97$ on an unseen geometry and unseen load, compared to $R2 approx 0.01$--$0.86$ for conventional models.
  arXiv  Detail & Related papers  (2026-06-06T18:17:08Z)
• Closed-Form Spectral Regularization for Multi-Task Model Merging [96.82449201305234]
  Model merging combines several independently fine-tuned experts into a single multi-task model without any training data.<n>State-of-the-art merging methods formulate merging as a layer-wise interference problem.<n>We propose SWUDI, a closed-form method that combines a soft exponential filter, which matches the gradient-flow trajectory of iterative descent.
  arXiv  Detail & Related papers  (2026-06-05T14:00:47Z)
• Lighter-X: An Efficient and Plug-and-play Strategy for Graph-based Recommendation through Decoupled Propagation [49.865020394064096]
  We propose textbfLighter-X, an efficient and modular framework that can be seamlessly integrated with existing GNN-based recommender architectures.<n>Our approach substantially reduces both parameter size and computational complexity while preserving the theoretical guarantees and empirical performance of the base models.<n>Experiments demonstrate that Lighter-X achieves comparable performance to baseline models with significantly fewer parameters.
  arXiv  Detail & Related papers  (2025-10-11T08:33:08Z)
• Test time training enhances in-context learning of nonlinear functions [51.56484100374058]
  Test-time training (TTT) enhances model performance by explicitly updating designated parameters prior to each prediction.<n>We investigate the combination of TTT with in-context learning (ICL), where the model is given a few examples from the target distribution at inference time.
  arXiv  Detail & Related papers  (2025-09-30T03:56:44Z)
• From Editor to Dense Geometry Estimator [77.21804448599009]
  We introduce textbfFE2E, a framework that adapts an advanced editing model based on Diffusion Transformer (DiT) architecture for dense geometry prediction.<n>FE2E achieves over 35% performance gains on the ETH3D dataset and outperforms the DepthAnything series, which is trained on 100$times$ data.
  arXiv  Detail & Related papers  (2025-09-04T15:58:50Z)
• Turbocharging Gaussian Process Inference with Approximate Sketch-and-Project [12.874219128885064]
  We propose an approximate, distributed, accelerated sketch-and-project algorithm ($texttADASAP$) for solving linear systems.<n>We use the theory of determinantal point processes to show that the posterior mean induced by sketch-and-project rapidly converges to the true posterior mean.<n>$texttADASAP$ scales to a dataset with $> 3 cdot 108$ samples, a feat which has not been accomplished in the literature.
  arXiv  Detail & Related papers  (2025-05-19T20:46:26Z)
• Symmetry-Breaking Descent for Invariant Cost Functionals [0.0]
  We study the problem of reducing a task cost functional $W : Hs(M) to mathbbR$, not assumed continuous or differentiable.<n>We show that symmetry-breaking deformations of the signal can reduce the cost.
  arXiv  Detail & Related papers  (2025-05-19T15:06:31Z)
• Near-Optimal Online Learning for Multi-Agent Submodular Coordination: Tight Approximation and Communication Efficiency [52.60557300927007]
  We present a $textbfMA-OSMA$ algorithm to transfer the discrete submodular problem into a continuous optimization.<n>We also introduce a projection-free $textbfMA-OSEA$ algorithm, which effectively utilizes the KL divergence by mixing a uniform distribution.<n>Our algorithms significantly improve the $(frac11+c)$-approximation provided by the state-of-the-art OSG algorithm.
  arXiv  Detail & Related papers  (2025-02-07T15:57:56Z)
• Oblivious Stochastic Composite Optimization [47.48197617884748]
  We show that our algorithms converge without any prior knowledge on the parameters of the problem.<n>All three algorithms work without prior knowledge of the diameter of the feasible set, the Lipschitz constant or smoothness of the objective function.<n>We extend our framework to relative scale and demonstrate the efficiency and robustness of our methods on large-scale semidefinite programs.
  arXiv  Detail & Related papers  (2023-06-30T08:34:29Z)
• Arch-Graph: Acyclic Architecture Relation Predictor for Task-Transferable Neural Architecture Search [96.31315520244605]
  Arch-Graph is a transferable NAS method that predicts task-specific optimal architectures. We show Arch-Graph's transferability and high sample efficiency across numerous tasks. It is able to find top 0.16% and 0.29% architectures on average on two search spaces under the budget of only 50 models.
  arXiv  Detail & Related papers  (2022-04-12T16:46:06Z)
• Decentralized Sparse Linear Regression via Gradient-Tracking: Linear Convergence and Statistical Guarantees [23.256961881716595]
  We study a sparse linear regression over a network of agents, modeled as an undirected graph and no server node.<n>We analyze the convergence rate and statistical guarantees of a distributed projected gradient tracking-based algorithm.
  arXiv  Detail & Related papers  (2022-01-21T01:26:08Z)
• Last iterate convergence of SGD for Least-Squares in the Interpolation regime [19.05750582096579]
  We study the noiseless model in the fundamental least-squares setup. We assume that an optimum predictor fits perfectly inputs and outputs $langle theta_*, phi(X) rangle = Y$, where $phi(X)$ stands for a possibly infinite dimensional non-linear feature map.
  arXiv  Detail & Related papers  (2021-02-05T14:02:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.