Information Routing in Atomistic Foundation Models: How Task Alignment and Equivariance Shape Linear Disentanglement

• URL: http://arxiv.org/abs/2603.03155v2
• Date: Mon, 09 Mar 2026 06:36:19 GMT
• Title: Information Routing in Atomistic Foundation Models: How Task Alignment and Equivariance Shape Linear Disentanglement
• Abstract summary: We introduce Compositional Probe Decomposition (CPD), which linearly projects out composition signal and measures how much geometric information remains accessible to a Ridge probe.<n>We find a emphlinear accessibility gradient: models differ by $6.6times$ in geometric information accessible after composition removal.<n>We also show that nonlinear probes produce misleading results on residualized representations, recovering $R2 = 0.68$--$0.95$ on a purely compositional target, and recommend linear probes for this setting.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: What determines whether a molecular property prediction model organizes its representations so that geometric and compositional information can be cleanly separated? We introduce Compositional Probe Decomposition (CPD), which linearly projects out composition signal and measures how much geometric information remains accessible to a Ridge probe. We validate CPD with four independent checks, including a structural isomer benchmark where compositional projections score at chance while geometric residuals reach 94.6\% pairwise classification accuracy. Across ten models from five architectural families on QM9, we find a \emph{linear accessibility gradient}: models differ by $6.6\times$ in geometric information accessible after composition removal ($R^2_{\mathrm{geom}}$ from 0.081 to 0.533 for HOMO-LUMO gap). Three factors explain this gradient. Task alignment dominates: models trained on HOMO-LUMO gap ($R^2_{\mathrm{geom}}$ 0.44--0.53) outscore energy-trained models by $\sim$0.25 $R^2$ regardless of architecture. Within-architecture ablations on two independent architectures confirm this: PaiNN drops from 0.53 to 0.31 when retrained on energy, and MACE drops from 0.44 to 0.08. Data diversity partially compensates for misaligned objectives, with MACE pretrained on MPTraj (0.36) outperforming QM9-only energy models. Inside MACE's representations, information routes by symmetry type: $L{=}1$ (vector) channels preferentially encode dipole moment ($R^2 = 0.59$ vs.\ 0.38 in $L{=}0$), while $L{=}0$ (scalar) channels encode HOMO-LUMO gap ($R^2 = 0.76$ vs.\ 0.34 in $L{=}1$). This pattern is absent in ViSNet. We also show that nonlinear probes produce misleading results on residualized representations, recovering $R^2 = 0.68$--$0.95$ on a purely compositional target, and recommend linear probes for this setting.

Related papers

• CyFM: Cylindrical Optimal Transport for Few-Step Complex-Valued Flow Matching [0.0]
  Complex-valued signals, such as Magnetic Resonance Imaging (MRI) and audio spectrograms, are almost always modelled as flat two-channel Euclidean data.<n>We replace the inherited metric $dA2 + A2 d2$ by the decoupled product metric $dA2 + d2$ and measure what that substitution costs and what it buys.<n>We analyze Cylindrical Flow Matching (CyFM), which strictly bounds the regression target, and couple noise and data by exact minibatch Optimal Transport computed jointly over whole fields in the cylindrical metric.
  arXiv  Detail & Related papers  (2026-09-12T22:12:33Z)
• Exact Limits of Random Projections for Preserving Geometry: Distance Recovery, Nearest-Neighbor Rankings, and Covariance Shape in Gaussian Models [0.0]
  Johnson-Lindenstrauss lemma guarantees that a random projection of $n$ points to $m=O(varepsilon-2log n)$ dimensions preserves linear squared distances.<n>In high dimensions, however, distances concentrate around a baseline while key geometric information lies in much smaller fluctuations.<n>We show that the JL bound can therefore be uninformative about retained geometry.
  arXiv  Detail & Related papers  (2026-09-02T06:12:34Z)
• Round-Trip Consistency: Bidirectional Diffusion Models Can Predict Their Own Rollout Errors [51.56484100374058]
  Autoregressive models accumulate error over long rollouts, yet at deployment there is no ground truth to measure it against.<n>We train a single conditional latent diffusion model that steps a dynamical system forward or backward in time via a direction flag.<n>We show that this bidirectionality supplies a measurement-free test-time error signal.
  arXiv  Detail & Related papers  (2026-08-01T13:49:46Z)
• Level-Crossing Density as a Mesh-Free High-Frequency Auxiliary Loss for Implicit Neural Representations [0.0]
  We derive smooth Monte-Carlo estimators for a continuous neural field using no grid, no complex, no persistence.<n>The estimator is accurate to 1-3% against exact topology in 2D and 3D, and costs about 3 ms per iteration.<n>In 3D neural-SDF fitting, however, a failure mode we believe general to any sampled soft topology objective appears.
  arXiv  Detail & Related papers  (2026-07-07T04:23:23Z)
• CaricHarmony: Contrastive Diffusion Paths for Identity-Preserving Caricature Synthesis [49.596677723190886]
  Sketch-based caricature synthesis suffers from a fundamental failure mode.<n>Identity and shape conditions are combined in diffusion models, causing collapse toward bland portraits or unrecognizable distortions.<n>We present CaricHarmony, the first training-free method that explicitly resolves this contamination through parallel uncontaminated diffusion paths.
  arXiv  Detail & Related papers  (2026-06-11T22:57:59Z)
• Efficient Mean Curvature Computation on High-Dimensional Data Manifolds [52.452902154360565]
  Estimating local mean curvature at each point of a high-dimensional dataset is a key ingredient of machine learning algorithms.<n>This paper introduces two complementary contributions that together reduce this cost by several orders of magnitude.<n> Experiments on real-world datasets confirm speedups of 50 to 300 times relative to the original implementation.
  arXiv  Detail & Related papers  (2026-06-04T16:04:31Z)
• UR-JEPA: Uniform Rectifiability as a Regularizer for Joint-Embedding Predictive Architectures [0.0]
  We propose emphUR-JEPA, which targets a uniformly $n$-rectifiable measure of local dimension tangent $n$ at small scales.<n>We show that on all four datasets UR--JEPA($mathcalLtextCGLT$) produces a global geometric spectrum with a $4$ to $5$ order-of-magnitude drop at index $sim 20$ to $25$ out of $D = 32$, while LeJEPA's spectrum is near-flat (top-to-bottom ratio at most $3.6$)
  arXiv  Detail & Related papers  (2026-05-31T20:26:54Z)
• The Geometry of Forgetting: Temporal Knowledge Drift as an Independent Axis in LLM Representations [50.43168858368539]
  Large language models confidently produce outdated answers, and no existing method can detect them.<n>We show this is not an engineering failure but a structural one: temporal drift is encoded as a direction in the residual stream geometrically to both correctness and uncertainty.
  arXiv  Detail & Related papers  (2026-05-09T22:27:31Z)
• INC: An Indirect Neural Corrector for Auto-Regressive Hybrid PDE Solvers [61.84396402100827]
  We propose the Indirect Neural Corrector ($mathrmINC$), which integrates learned corrections into the governing equations.<n>$mathrmINC$ reduces the error amplification on the order of $t-1 + L$, where $t$ is the timestep and $L$ the Lipschitz constant.<n>We test $mathrmINC$ in extensive benchmarks, covering numerous differentiable solvers, neural backbones, and test cases ranging from a 1D chaotic system to 3D turbulence.
  arXiv  Detail & Related papers  (2025-11-16T20:14:28Z)
• Bounds on Lorentz-violating parameters in magnetically confined 2D systems: A phenomenological approach [0.0]
  We present a framework to constrain minimal SME coefficients $a_mu$ and $b_mu$ using magnetically confined two-dimensional electron systems.<n>Working in the nonrelativistic (Schr"odinger--Pauli) limit with effective mass, we derive the radial problem for cylindrical geometries.
  arXiv  Detail & Related papers  (2025-10-28T11:11:59Z)
• Learning single-index models via harmonic decomposition [21.065469907392643]
  We study the problem of learning single-index models, where the label $y in mathbbR$ depends on the input $boldsymbolx in mathbbRd$ only through an unknown one-dimensional projection.<n>We introduce two families of estimators -- based on tensor unfolding and online SGD -- that respectively achieve either optimal sample complexity or optimal runtime.
  arXiv  Detail & Related papers  (2025-06-11T15:59:53Z)
• The Spacetime of Diffusion Models: An Information Geometry Perspective [40.23096112113255]
  We show that the standard pullback approach, utilizing the deterministic probability flow ComplementODE decoder, is fundamentally flawed.<n>We introduce a latent spacetime $z=(x_t,t)$ that indexes the family of denoising distributions $p(x_t,t)$ across all noise scales.<n>The resulting structure induces a principled Diffusion Distance Edit, where geodesics trace minimal sequences of noise and denoise edits between data.
  arXiv  Detail & Related papers  (2025-05-23T06:16:58Z)
• Symmetry-Breaking Descent for Invariant Cost Functionals [0.0]
  We study the problem of reducing a task cost functional $W : Hs(M) to mathbbR$, not assumed continuous or differentiable.<n>We show that symmetry-breaking deformations of the signal can reduce the cost.
  arXiv  Detail & Related papers  (2025-05-19T15:06:31Z)
• Robustness of Nonlinear Representation Learning [60.15898117103069]
  We study the problem of unsupervised representation learning in slightly misspecified settings.<n>We show that the mixing can be identified up to linear transformations and small errors.<n>Those results are a step towards identifiability results for unsupervised representation learning for real-world data.
  arXiv  Detail & Related papers  (2025-03-19T15:57:03Z)
• Learning Orthogonal Multi-Index Models: A Fine-Grained Information Exponent Analysis [54.57279006229212]
  Information exponent has played an important role in predicting the sample complexity of online gradient descent.<n>In this work, we show that by considering both second- and higher-order terms, we can first learn the relevant space using the second-order terms.<n>The overall sample and complexity of online SGD is $tildeO( d PL-1 )$.
  arXiv  Detail & Related papers  (2024-10-13T00:14:08Z)
• Implicit Hypersurface Approximation Capacity in Deep ReLU Networks [0.0]
  We develop a geometric approximation theory for deep feed-forward neural networks with ReLU activations. We show that a deep fully-connected ReLU network of width $d+1$ can implicitly construct an approximation as its zero contour.
  arXiv  Detail & Related papers  (2024-07-04T11:34:42Z)
• Neural network learns low-dimensional polynomials with SGD near the information-theoretic limit [75.4661041626338]
  We study the problem of gradient descent learning of a single-index target function $f_*(boldsymbolx) = textstylesigma_*left(langleboldsymbolx,boldsymbolthetarangleright)$<n>We prove that a two-layer neural network optimized by an SGD-based algorithm learns $f_*$ with a complexity that is not governed by information exponents.
  arXiv  Detail & Related papers  (2024-06-03T17:56:58Z)
• Computational-Statistical Gaps in Gaussian Single-Index Models [77.1473134227844]
  Single-Index Models are high-dimensional regression problems with planted structure. We show that computationally efficient algorithms, both within the Statistical Query (SQ) and the Low-Degree Polynomial (LDP) framework, necessarily require $Omega(dkstar/2)$ samples.
  arXiv  Detail & Related papers  (2024-03-08T18:50:19Z)
• Effective Minkowski Dimension of Deep Nonparametric Regression: Function Approximation and Statistical Theories [70.90012822736988]
  Existing theories on deep nonparametric regression have shown that when the input data lie on a low-dimensional manifold, deep neural networks can adapt to intrinsic data structures. This paper introduces a relaxed assumption that input data are concentrated around a subset of $mathbbRd$ denoted by $mathcalS$, and the intrinsic dimension $mathcalS$ can be characterized by a new complexity notation -- effective Minkowski dimension.
  arXiv  Detail & Related papers  (2023-06-26T17:13:31Z)
• Inverting brain grey matter models with likelihood-free inference: a tool for trustable cytoarchitecture measurements [62.997667081978825]
  characterisation of the brain grey matter cytoarchitecture with quantitative sensitivity to soma density and volume remains an unsolved challenge in dMRI. We propose a new forward model, specifically a new system of equations, requiring a few relatively sparse b-shells. We then apply modern tools from Bayesian analysis known as likelihood-free inference (LFI) to invert our proposed model.
  arXiv  Detail & Related papers  (2021-11-15T09:08:27Z)
• Minimax Optimal Regression over Sobolev Spaces via Laplacian Eigenmaps on Neighborhood Graphs [25.597646488273558]
  We study the statistical properties of Principal Components Regression with Laplacian Eigenmaps (PCR-LE) PCR-LE works by projecting a vector of observed responses onto a subspace spanned by certain eigenvectors of a neighborhood graph Laplacian. We show that PCR-LE achieves minimax rates of convergence for random design regression over Sobolev spaces.
  arXiv  Detail & Related papers  (2021-11-14T17:32:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.