Out-of-distribution Neural Inference in Dynamical Ising Models

• URL: http://arxiv.org/abs/2607.03039v1
• Date: Fri, 03 Jul 2026 07:31:30 GMT
• Title: Out-of-distribution Neural Inference in Dynamical Ising Models
• Abstract summary: We show that data-driven training produces distinct statistical strategies under topology and temperature shifts.<n>High in-distribution accuracy and apparent out-of-distribution do not necessarily imply a learned dynamics-to-structure rule.<n>Our results identify a concrete failure mode of standard data-driven learning in physical inverse problems.
• Score: 0.30586855806896035
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural networks are increasingly used to infer hidden physical structure from dynamical observations, yet it remains unclear whether their out-of-distribution performance reflects transferable physical rule learning. We address this question in a controlled inverse problem: reconstructing interaction graphs of a kinetic Ising model from Glauber magnetization trajectories. Across convolutional, graph, Transformer, and hybrid architectures, we find that data-driven training produces distinct and reproducible statistical strategies under topology and temperature shifts. Edge-population diagnostics reveal that Transformer-based models tend to preserve the link density of the training ensemble, whereas convolutional models can collapse toward sparse- or no-link predictions that appear out-of-distribution stable by exploiting the majority no-link class. Thus, high in-distribution accuracy and apparent out-of-distribution robustness do not necessarily imply a learned dynamics-to-structure rule. Instead, neural reconstruction can be governed by architecture-dependent statistical priors. Our results identify a concrete failure mode of standard data-driven learning in physical inverse problems and motivate rule-guided principles for machine-learning-assisted scientific discovery.

Related papers

• Dynamics and Representation Structure of Local Approximations to Gradient-Based Learning in Linear Recurrent Neural Networks [56.42974363744522]
  We study the learning dynamics of random feedback local online (RFLO) and truncated backpropagation through time (tBPTT)<n>Our work provides analytical insight into how locality constraints shape learning dynamics, with implications for neuroscientific models of learning and alternative optimization approaches for RNNs.
  arXiv  Detail & Related papers  (2026-05-29T18:19:45Z)
• The Interplay of Data Structure and Imbalance in the Learning Dynamics of Diffusion Models [8.547042933482645]
  We study class-dependent learning in score-based diffusion models.<n>We reveal the explicit hierarchy governing these dynamics.<n>We validate our theoretical predictions empirically using U-Net models trained on Fashion MNIST.
  arXiv  Detail & Related papers  (2026-05-07T14:44:53Z)
• Learning a Generative Meta-Model of LLM Activations [75.30161960337892]
  We create "meta-models" that learn the distribution of a network's internal states.<n>Applying the meta-model's learned prior to steering interventions improves fluency, with larger gains as loss decreases.<n>These results suggest generative meta-models offer a scalable path toward interpretability without restrictive structural assumptions.
  arXiv  Detail & Related papers  (2026-02-06T18:59:56Z)
• Transformer Is Inherently a Causal Learner [27.79148022495734]
  We show that transformer trained in an autoregressive manner naturally encodes time-delayed causal structures.<n>We prove this connection theoretically under standard identifiability conditions.<n>This approach greatly surpasses the performance of state-of-the-art discovery algorithms.
  arXiv  Detail & Related papers  (2026-01-09T09:10:04Z)
• Spatially-informed transformers: Injecting geostatistical covariance biases into self-attention for spatio-temporal forecasting [0.0]
  We propose a hybrid architecture that injects a geostatistic inductive bias directly into the decomposing self-attention mechanism via a learnable costatistics kernel.<n>We demonstrate the phenomenon of Deep Variography'', where the network successfully recovers the true spatial parameters of the underlying process end-to-end via backpropagation.
  arXiv  Detail & Related papers  (2025-12-19T15:32:24Z)
• Topology Identification and Inference over Graphs [61.06365536861156]
  Topology identification and inference of processes evolving over graphs arise in timely applications involving brain, transportation, financial, power, as well as social and information networks.<n>This chapter provides an overview of graph topology identification and statistical inference methods for multidimensional data.
  arXiv  Detail & Related papers  (2025-12-11T00:47:09Z)
• Towards agent-based-model informed neural networks [0.5787117733071417]
  We present a framework for designing neural networks consistent with the underlying principles of agent-based models.<n>We validate the framework across three case studies of increasing complexity.
  arXiv  Detail & Related papers  (2025-12-05T14:50:50Z)
• Optimal Linear Baseline Models for Scientific Machine Learning [37.69303106863453]
  We develop a unified theoretical framework for analyzing linear encoder-decoder architectures.<n>We derive closed-form, rank-constrained linear optimal mappings for forward modeling and inverse recovery tasks.<n>This work provides a robust baseline for understanding and benchmarking learned neural network models for scientific machine learning problems.
  arXiv  Detail & Related papers  (2025-08-07T20:17:07Z)
• The Edge-of-Reach Problem in Offline Model-Based Reinforcement Learning [31.8260779160424]
  We investigate how popular algorithms perform as the learned dynamics model is improved.<n>We propose Reach-Aware Learning (RAVL), a simple and robust method that directly addresses the edge-of-reach problem.
  arXiv  Detail & Related papers  (2024-02-19T20:38:00Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Prequential MDL for Causal Structure Learning with Neural Networks [9.669269791955012]
  We show that the prequential minimum description length principle can be used to derive a practical scoring function for Bayesian networks. We obtain plausible and parsimonious graph structures without relying on sparsity inducing priors or other regularizers which must be tuned. We discuss how the the prequential score relates to recent work that infers causal structure from the speed of adaptation when the observations come from a source undergoing distributional shift.
  arXiv  Detail & Related papers  (2021-07-02T22:35:21Z)
• Gradient Starvation: A Learning Proclivity in Neural Networks [97.02382916372594]
  Gradient Starvation arises when cross-entropy loss is minimized by capturing only a subset of features relevant for the task. This work provides a theoretical explanation for the emergence of such feature imbalance in neural networks.
  arXiv  Detail & Related papers  (2020-11-18T18:52:08Z)
• Network Diffusions via Neural Mean-Field Dynamics [52.091487866968286]
  We propose a novel learning framework for inference and estimation problems of diffusion on networks. Our framework is derived from the Mori-Zwanzig formalism to obtain an exact evolution of the node infection probabilities. Our approach is versatile and robust to variations of the underlying diffusion network models.
  arXiv  Detail & Related papers  (2020-06-16T18:45:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.