Susceptibilities and Patterning: A Primer on Linear Response in Bayesian Learning

• URL: http://arxiv.org/abs/2605.07980v1
• Date: Fri, 08 May 2026 16:43:48 GMT
• Title: Susceptibilities and Patterning: A Primer on Linear Response in Bayesian Learning
• Abstract summary: Notes introduce the theory of susceptibilities as developed in [arXiv:2504.18274, arXiv:2601.12703] for interpreting neural networks.<n>We motivate the theory from its statistical-mechanical foundations, then give a detailed exposition of susceptibilities, their empirical estimators, and their connection to the geometry of the loss landscape.
• Score: 1.5500856913141685
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: These notes introduce the theory of susceptibilities as developed in [arXiv:2504.18274, arXiv:2601.12703] for interpreting neural networks. The susceptibility of an observable $φ$ to a data perturbation is defined as a derivative of a posterior expectation, which by the fluctuation--dissipation theorem equals a posterior covariance. Different choices of $φ$ yield different objects: per-sample losses give the influence matrix (the Bayesian influence function of [arXiv:2509.26544]), while component-localized observables give the structural susceptibility matrix that pairs model components with data patterns. The susceptibility matrix is (up to a factor of $nβ$) the Jacobian of the map from data distributions to structural coordinates; its pseudo-inverse provides a linearized solution to the patterning problem of [arXiv:2601.13548]: finding data perturbations that produce a desired structural change. We motivate the theory from its statistical-mechanical foundations, then give a detailed exposition of susceptibilities, their empirical estimators, and their connection to the geometry of the loss landscape.

Related papers

• A Random Matrix Theory Perspective on the Consistency of Diffusion Models [31.63433424187031]
  Diffusion models trained on different subsets of a dataset often produce strikingly similar outputs when given the same noise seed.<n>We develop a random matrix theory (RMT) framework that quantifies how finite shape the expectation and variance of the learned denoiser and sampling map.<n>We validate its predictions on UNet and DiT architectures in their non-memorization regime.
  arXiv  Detail & Related papers  (2026-02-02T23:30:28Z)
• Theory and computation for structured variational inference [13.739142487196034]
  We prove the first results for existence, uniqueness, and self-consistency of the variational approximation.<n>We develop a gradient-based algorithm with provable guarantees for computing the variational approximation.
  arXiv  Detail & Related papers  (2025-11-13T02:57:54Z)
• Loss-Complexity Landscape and Model Structure Functions [53.92822954974537]
  We develop a framework for dualizing the Kolmogorov structure function $h_x(alpha)$.<n>We establish a mathematical analogy between information-theoretic constructs and statistical mechanics.<n>We explicitly prove the Legendre-Fenchel duality between the structure function and free energy.
  arXiv  Detail & Related papers  (2025-07-17T21:31:45Z)
• Landscape Complexity for the Empirical Risk of Generalized Linear Models: Discrimination between Structured Data [2.486161976966064]
  We use the Kac-Rice formula and results from random matrix theory to obtain the average number of critical points of a family of high-dimensional empirical loss functions.<n>The correlations are introduced to model the existence of structure in the data, as is common in current Machine-Learning systems.<n>For completeness, we also treat the case of a loss function used in training Generalized Linear Models in the presence of correlated input data.
  arXiv  Detail & Related papers  (2025-03-18T16:44:33Z)
• Induced Covariance for Causal Discovery in Linear Sparse Structures [55.2480439325792]
  Causal models seek to unravel the cause-effect relationships among variables from observed data. This paper introduces a novel causal discovery algorithm designed for settings in which variables exhibit linearly sparse relationships.
  arXiv  Detail & Related papers  (2024-10-02T04:01:38Z)
• Towards a mathematical understanding of learning from few examples with nonlinear feature maps [68.8204255655161]
  We consider the problem of data classification where the training set consists of just a few data points. We reveal key relationships between the geometry of an AI model's feature space, the structure of the underlying data distributions, and the model's generalisation capabilities.
  arXiv  Detail & Related papers  (2022-11-07T14:52:58Z)
• An Indirect Rate-Distortion Characterization for Semantic Sources: General Model and the Case of Gaussian Observation [83.93224401261068]
  Source model is motivated by the recent surge of interest in the semantic aspect of information. intrinsic state corresponds to the semantic feature of the source, which in general is not observable. Rate-distortion function is the semantic rate-distortion function of the source.
  arXiv  Detail & Related papers  (2022-01-29T02:14:24Z)
• Structural Causal Models Are (Solvable by) Credal Networks [70.45873402967297]
  Causal inferences can be obtained by standard algorithms for the updating of credal nets. This contribution should be regarded as a systematic approach to represent structural causal models by credal networks. Experiments show that approximate algorithms for credal networks can immediately be used to do causal inference in real-size problems.
  arXiv  Detail & Related papers  (2020-08-02T11:19:36Z)
• Accounting for Unobserved Confounding in Domain Generalization [107.0464488046289]
  This paper investigates the problem of learning robust, generalizable prediction models from a combination of datasets. Part of the challenge of learning robust models lies in the influence of unobserved confounders. We demonstrate the empirical performance of our approach on healthcare data from different modalities.
  arXiv  Detail & Related papers  (2020-07-21T08:18:06Z)
• Sparse Cholesky covariance parametrization for recovering latent structure in ordered data [1.5349431582672617]
  We focus on arbitrary zero patterns in the Cholesky factor of a covariance matrix. For the ordered scenario, we propose a novel estimation method that is based on matrix loss penalization. We give guidelines, based on the empirical results, about which of the methods analysed is more appropriate for each setting.
  arXiv  Detail & Related papers  (2020-06-02T08:35:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.