Saturating Scaling Laws for Equational Discovery: A Phenomenology of Growth Dynamics in Three Toy Substrates with Two Real-World Replications

• URL: http://arxiv.org/abs/2605.23983v1
• Date: Thu, 14 May 2026 21:37:29 GMT
• Title: Saturating Scaling Laws for Equational Discovery: A Phenomenology of Growth Dynamics in Three Toy Substrates with Two Real-World Replications
• Abstract summary: We investigate growth dynamics in deterministic equational discovery substrates.<n>Short-range substrate sizes fit a power-law N(t) proportional to tb.<n>A saturating power-law dN/dt = K Nk exp(-mu N) of which the pure power-law is the short-range approximation.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We investigate growth dynamics in deterministic equational discovery substrates. Across three toy domains (arithmetic, boolean, higher-order list; n=592 trajectories), short-range substrate sizes fit a power-law N(t) proportional to t^b. Within each substrate b is architecture-sensitive (cross-validated R^2 approximately 0.82); the regression does not transfer across substrates (arith+bool to list yields R^2 approximately -0.84). A heuristic mean-field closure model predicts a saturating power-law dN/dt = K N^k exp(-mu N) of which the pure power-law is the short-range approximation. Three robustness checks: bootstrap intervals on (k, mu) are tight in 4/5 toy trajectories and degenerate in 1/5; out-of-sample forecasting on toy data (fit first 100 epochs, predict next 400) is won by pure power-law 5/5, indicating the toy trajectories do not reach saturation; on two real-world growth proxies the result splits. New Mathlib/*.lean file additions per month (mathlib4, 60 months, 9701 files) support the saturating form on OOS forecasting by approximately 7x over pure power-law; Coq mathcomp monthly commits (129 months, 3083 commits) favour pure power-law on both tests with mu collapsing to zero. The dynamics are substrate-conditional at two levels: within-substrate architecture-to-b regressions do not transfer, and the preferred functional family for N(t) itself (pure vs. saturating power-law) differs by substrate. We propose "saturating power-law growth with substrate-conditional (k, mu), observable when the substrate has reached its saturation regime" as a working framing.

Related papers

• On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability [57.03393882933515]
  We describe the architecture and ablations of Qwen3.8-Flash-Next, a sparse mixture-of-experts model with 125B parameters, 6B activated per token, and 51B parameters of n-gram embedding tables held off the accelerator.<n>On fourteen pre-training benchmarks the model leads the 397B-A17B predecessor on eight and trails it on the rest by at most 2.6 points, at 1/3 the activated parameters, 1/3 the training tokens, and roughly 1/9 the training FLOPs.
  arXiv  Detail & Related papers  (2026-08-31T06:35:07Z)
• SANE: State Anomaly Neutralization for Stable Extreme-Context Delta-Rule Models [53.86918766240095]
  Delta-Rule recurrent models maintain a fixed-size state, enabling $O(1)$ inference memory but potentially becoming unstable under extreme-context extrapolation.<n>We propose textbfState Anomaly Neutralization (SANE), which applies adaptive $tanh$ compression at chunk boundaries while preserving the intra-chunk parallel structure.
  arXiv  Detail & Related papers  (2026-08-23T10:41:07Z)
• No Subspace to Track: Non-Identifiability and Optimizer State in Low-Rank Training [0.0]
  Memory-efficients such as GaLore train large language models by projecting gradients onto a rank-r subspace.<n>We show that beyond a small reproducible core, there is no such object.
  arXiv  Detail & Related papers  (2026-07-07T06:06:04Z)
• The Complexity Ceiling Benchmark: A Multi-Domain Evaluation of Sequential Reasoning Under Depth Scaling [8.616356693448987]
  We introduce the Complexity Ceiling Benchmark (CCB), a controlled evaluation of how language-model reasoning decays.<n>CCB fixes the semantic content of a task and varies only its depth N in 5,...,50 across three structurally distinct regimes.<n>We find a consistent pattern of geometric per-step decay with widely separated domain ceilings.
  arXiv  Detail & Related papers  (2026-06-28T08:56:34Z)
• Decomposing Evolutionary Mixture-of-LoRA Architectures: The Routing Lever, the Lifecycle Penalty, and a Substrate-Conditional Boundary [3.151184728006369]
  We decompose an evolutionary mixtureofLoRA system into three factors -- a router rewrite, a perdomain leave-one-out evaluation scope, and a lifecycle of death plus alpha-blend inheritance plus SVD mutation plus slot reallocation.<n>A synthetic sandbox locates a substrate-conditional regime: evolutionary search on the routing channel is load-bearing only when are pre-aligned to the task; in every other regime tested it underperforms, ties, or actively degrades the solution.
  arXiv  Detail & Related papers  (2026-05-11T18:59:11Z)
• THEIA: Learning Complete Kleene Three-Valued Logic in a Pure-Neural Modular Architecture [0.0]
  THEIA is a 2.75M modular neural architecture that learns the complete Kleene three-valued logic (K3) truth table from task data without external symbolic inference or hand-encoded K3 gate primitives.<n> Transformer baselines also reach >99% on all 39 rules, and flats match THEIA on Phase-1 accuracy within 0.04pp.
  arXiv  Detail & Related papers  (2026-04-13T10:44:15Z)
• FluidWorld: Reaction-Diffusion Dynamics as a Predictive Substrate for World Models [0.0]
  Current approaches default to Transformer-based predictors operating in learned latent spaces.<n>This paper asks a foundational question: is self-attention necessary for predictive world modeling, or can alternative computational substrates achieve comparable or superior results?<n>I introduce FluidWorld, a proof-of-concept world model whose predictive dynamics are governed by partial differential equations (PDEs) of reaction-diffusion type.
  arXiv  Detail & Related papers  (2026-03-22T16:49:43Z)
• BASIS: Balanced Activation Sketching with Invariant Scalars for "Ghost Backpropagation" [0.0]
  Activation memory required for exact backpropagation scales linearly with network depth, context length, and feature dimensionality.<n>In this paper, we introduce an efficient backpropagation algorithm that fully decouples activation memory from the batch and sequence dimensions.
  arXiv  Detail & Related papers  (2026-03-05T20:38:25Z)
• Unregularized Linear Convergence in Zero-Sum Game from Preference Feedback [50.89125374999765]
  We provide the first convergence guarantee for Optimistic Multiplicative Weights Update ($mathtOMWU$) in NLHF.<n>Our analysis identifies a novel marginal convergence behavior, where the probability of rarely played actions grows exponentially from exponentially small values.
  arXiv  Detail & Related papers  (2025-12-31T12:08:29Z)
• Temporal Zoom Networks: Distance Regression and Continuous Depth for Efficient Action Localization [6.908972852063454]
  Temporal action localization requires both precise boundary detection and computational efficiency.<n>We address this through two complementary innovations: Boundary Distance Regression (BDR) and Adaptive Temporal Refinement (ATR)<n>On THUMOS14, our method achieves 56.5% mAP@0.7 with 151G FLOPs, using 36% fewer FLOPs than ActionFormer++ (55.7% mAP@0.7 at 235G)
  arXiv  Detail & Related papers  (2025-11-06T00:41:54Z)
• SKI to go Faster: Accelerating Toeplitz Neural Networks via Asymmetric Kernels [69.47358238222586]
  Toeplitz Neural Networks (TNNs) are a recent sequence model with impressive results. We aim to reduce O(n) computational complexity and O(n) relative positional encoder (RPE) multi-layer perceptron (MLP) and decay bias calls. For bidirectional models, this motivates a sparse plus low-rank Toeplitz matrix decomposition.
  arXiv  Detail & Related papers  (2023-05-15T21:25:35Z)
• Revisiting Weighted Strategy for Non-stationary Parametric Bandits [82.1942459195896]
  This paper revisits the weighted strategy for non-stationary parametric bandits. We propose a refined analysis framework, which produces a simpler weight-based algorithm. Our new framework can be used to improve regret bounds of other parametric bandits.
  arXiv  Detail & Related papers  (2023-03-05T15:11:14Z)
• Bounding the Width of Neural Networks via Coupled Initialization -- A Worst Case Analysis [121.9821494461427]
  We show how to significantly reduce the number of neurons required for two-layer ReLU networks. We also prove new lower bounds that improve upon prior work, and that under certain assumptions, are best possible.
  arXiv  Detail & Related papers  (2022-06-26T06:51:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.