Why Do Few-Step Text Latents Fail When Image Latents Work? Non-Commitment at Sharp Categorical Readouts

• URL: http://arxiv.org/abs/2606.30705v1
• Date: Mon, 29 Jun 2026 14:20:57 GMT
• Title: Why Do Few-Step Text Latents Fail When Image Latents Work? Non-Commitment at Sharp Categorical Readouts
• Abstract summary: We show that a smooth, regularity-limited deterministic map cannot resolve a discrete branch choice before a sharp categorical readout.<n>In the overlapping regime of real text autoencoders, we prove that the posterior-mean terminal step flips tokens at the rate of the latent mass in an $O(s(t)$ tube around decision boundaries.
• Score: 1.1760370831826417
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deterministic few-step generation succeeds on continuous image latents but collapses to incoherent text on continuous text latents, and we show the cause is geometric rather than a training or scaling deficiency: a smooth, regularity-limited deterministic map cannot resolve a discrete branch choice before a sharp categorical readout, so few-step failure is governed by decoder sharpness, not transport accuracy. In the overlapping regime of real text autoencoders, we prove (Theorem 3) that the posterior-mean terminal step flips tokens at the rate of the latent mass in an $O(s(t))$ tube around decision boundaries. Two diagnostics, DABI (readout sharpness) and CCI (categorical commitment), measured on published checkpoints show that four independently built continuous-text decoders amplify a boundary-aligned perturbation far beyond a norm-matched isotropic one (DABI from $5\times10^{2}$ to $>10^{5}$), while image decoders have DABI $\approx 1$. Two mechanisms escape the continuous bound: categorical commitment (autoregressive decoders succeed despite sharper readouts) and stochastic re-injection (deterministic ODE at $K=4$ gives PPL 294 versus SDE 50 on the same model). In the idealized separated regime we prove matching sharp transport laws, including a dimension phase diagram: the deterministic stiffness needed to separate $M$ modes grows as $Θ(\sqrt{\log M})$ once the latent dimension is $Ω(\log M)$ (and as $M^{1/n}$ in fixed dimension), with a depth-$B$ hierarchy giving a $\sqrt{B}$-smaller per-step peak (Theorems 5-7); a coarea identity links these to the overlapping tube (Theorem 17). The result is an accuracy-depth-stiffness tradeoff: within the deterministic-continuous class the cost is irreducible, and both escapes step outside it.

Related papers

• Constrained Online Learning with Noisy Constraint Values [55.29259818039367]
  Under common feasibility, our LEDGER algorithm attains $O(sqrt T)$ expected regret and $O(sqrtTlog(eT))$ expected budget violation.<n>Neither a Slater condition, independence between feedback channels, nor an absolute constraint-value bound is needed.
  arXiv  Detail & Related papers  (2026-09-07T01:38:41Z)
• Drift Variation Autoencoder: Unifying Generation and Representation Learning through Conditional Posterior Flow Matching [0.0]
  This work takes the corresponding posterior $P(Xmid C)$ as the common statistical object for conditional generation.<n> Drift Variation autoencoder trains a masked encoder $Z=E(C)$ and a conditional flow decoder with one clean-prediction Flow Matching loss.
  arXiv  Detail & Related papers  (2026-08-25T20:38:50Z)
• SANE: State Anomaly Neutralization for Stable Extreme-Context Delta-Rule Models [53.86918766240095]
  Delta-Rule recurrent models maintain a fixed-size state, enabling $O(1)$ inference memory but potentially becoming unstable under extreme-context extrapolation.<n>We propose textbfState Anomaly Neutralization (SANE), which applies adaptive $tanh$ compression at chunk boundaries while preserving the intra-chunk parallel structure.
  arXiv  Detail & Related papers  (2026-08-23T10:41:07Z)
• Solver-Hard Is Not Model-Hard: A Hardness-Controlled Diagnostic for LLM Constraint Reasoning [0.0]
  LLM constraint reasoners are often evaluated near the random-SAT phase transition, confounding density and solver hardness.<n>We compare proof-hard expander-Tseitin and proof-easy ladder-Tseitin formulas, pigeonhole anchors, and density-mismatched controls.<n>Across three included models, the near-matched-density accuracy gaps range from $-32$ to $+20$ points.
  arXiv  Detail & Related papers  (2026-07-19T03:23:22Z)
• A Mathematical Theory of Value: a synthesis on goal-directed agency under resource constraints [6.057587531186626]
  We show that value, the quantity goal-directed agents create, destroy, and exchange, is a lawful structural quantity in the same category as information.<n>For populations, value is frame-relative while price is frame-independent; a fleet that pools its resource and fuses its perception inherits the ceiling.
  arXiv  Detail & Related papers  (2026-06-10T16:11:04Z)
• Metric--Phase Fields: Decoupling Distance and Sign for Thin-Structure Reconstruction from Unoriented Point Clouds [67.30236447078187]
  We introduce Metric-Phase Fields (MPFs), a decoupled implicit representation that separates metric proximity from topological phase.<n>Experiments on both synthetic and scanned thin-shell and thin-plate shapes demonstrate that MPFs preserve thin and layered structures more faithfully than recent SDF-based methods.
  arXiv  Detail & Related papers  (2026-05-25T07:08:57Z)
• Prism: Structural Symmetry Scanning via Duality-Constrained Laplacian Projection [0.0]
  We introduce textbfPrism, a framework for structural symmetry diagnosis in complex networks.<n>Prism computes the emphduality defect $(L,P) = |LP - PL|_F / |L|_F$ -- a scalar measuring how far the network deviates from structural self-consistency.
  arXiv  Detail & Related papers  (2026-05-18T04:20:05Z)
• Stateful Reasoning via Insight Replay [51.85629502016196]
  Chain-of-Thought (CoT) reasoning has become a foundation for eliciting multi-step reasoning in large language models.<n>We identify a major cause of this phenomenon: as the CoT grows, the model's attention to critical insights produced earlier in the trace gradually weakens.<n>We propose textbfInsightReplay, a stateful reasoning approach in which the model periodically extracts critical insights from its reasoning trace and replays them near the active generation frontier.
  arXiv  Detail & Related papers  (2026-05-14T06:52:59Z)
• Wavelet Variance Equipartition as a Threshold for World-Model Quality and Quantum Kernel TN-Simulability [0.0]
  We identify the wavelet scaling exponent $$ as a critical diagnostic.<n>We establish $= 1/2$ as a sharp transition boundary for the classical simulability of amplitude-encoded quantum kernels.<n>We prove this variance scales strictly as $Var[X] = (d-2)$.
  arXiv  Detail & Related papers  (2026-05-12T05:41:12Z)
• TIDES: Implicit Time-Awareness in Selective State Space Models [0.0]
  Continuous-time SSMs handle irregular timestamps, but their dynamics remain linear time-invariant (LTI)<n>We propose textbfTIDES, a selective SSM variant that reconciles selective and continuous architectures.<n> TIDES sets the new state-of-the-art average rank on UEA time-series classification and the Physiome-ODE regression benchmark.
  arXiv  Detail & Related papers  (2026-05-10T20:34:06Z)
• The Geometry of Forgetting: Temporal Knowledge Drift as an Independent Axis in LLM Representations [50.43168858368539]
  Large language models confidently produce outdated answers, and no existing method can detect them.<n>We show this is not an engineering failure but a structural one: temporal drift is encoded as a direction in the residual stream geometrically to both correctness and uncertainty.
  arXiv  Detail & Related papers  (2026-05-09T22:27:31Z)
• Spectral Sentinel: Scalable Byzantine-Robust Decentralized Federated Learning via Sketched Random Matrix Theory on Blockchain [0.0]
  Byzantine clients poison gradients under heterogeneous (Non-IID) data.<n>We propose Spectral Sentinel, a Byzantine detection and aggregation framework.<n>We implement the full system with blockchain integration on Polygon networks.
  arXiv  Detail & Related papers  (2025-12-14T09:43:03Z)
• Variance-Dependent Regret Bounds for Linear Bandits and Reinforcement Learning: Adaptivity and Computational Efficiency [90.40062452292091]
  We present the first computationally efficient algorithm for linear bandits with heteroscedastic noise. Our algorithm is adaptive to the unknown variance of noise and achieves an $tildeO(d sqrtsum_k = 1K sigma_k2 + d)$ regret. We also propose a variance-adaptive algorithm for linear mixture Markov decision processes (MDPs) in reinforcement learning.
  arXiv  Detail & Related papers  (2023-02-21T00:17:24Z)
• Horizon-Free and Variance-Dependent Reinforcement Learning for Latent Markov Decision Processes [62.90204655228324]
  We study regret minimization for reinforcement learning (RL) in Latent Markov Decision Processes (LMDPs) with context in hindsight. We design a novel model-based algorithmic framework which can be instantiated with both a model-optimistic and a value-optimistic solver.
  arXiv  Detail & Related papers  (2022-10-20T21:32:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.