Bulk-boundary decomposition of neural networks

• URL: http://arxiv.org/abs/2511.02003v1
• Date: Mon, 03 Nov 2025 19:18:20 GMT
• Title: Bulk-boundary decomposition of neural networks
• Authors: Donghee Lee, Hye-Sung Lee, Jaeok Yi,
• Abstract summary: We present the bulk-boundary decomposition as a new framework for understanding the training dynamics of deep neural networks.<n>As a natural extension, we develop a field-theoretic formulation of neural dynamics based on this decomposition.
• Score: 3.293822034705118
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present the bulk-boundary decomposition as a new framework for understanding the training dynamics of deep neural networks. Starting from the stochastic gradient descent formulation, we show that the Lagrangian can be reorganized into a data-independent bulk term and a data-dependent boundary term. The bulk captures the intrinsic dynamics set by network architecture and activation functions, while the boundary reflects stochastic interactions from training samples at the input and output layers. This decomposition exposes the local and homogeneous structure underlying deep networks. As a natural extension, we develop a field-theoretic formulation of neural dynamics based on this decomposition.

Related papers

• Revisiting Deep Information Propagation: Fractal Frontier and Finite-size Effects [16.42026346710591]
  We study information propagation in randomly neural networks with finite width and reveal that the boundary between ordered and chaotic regimes exhibits a fractal structure.<n>This shows the fundamental complexity of neural network dynamics, in a setting that is independent of input data and optimization.
  arXiv  Detail & Related papers  (2025-08-05T08:49:24Z)
• An Analytical Characterization of Sloppiness in Neural Networks: Insights from Linear Models [18.99511760351873]
  Recent experiments have shown that training trajectories of multiple deep neural networks evolve on a remarkably low-dimensional "hyper-ribbon-like" manifold.<n>Inspired by the similarities in the training trajectories of deep networks and linear networks, we analytically characterize this phenomenon for the latter.<n>We show that the geometry of this low-dimensional manifold is controlled by (i) the decay rate of the eigenvalues of the input correlation matrix of the training data, (ii) the relative scale of the ground-truth output to the weights at the beginning of training, and (iii) the number of steps of gradient descent.
  arXiv  Detail & Related papers  (2025-05-13T19:20:19Z)
• The Cooperative Network Architecture: Learning Structured Networks as Representation of Sensory Patterns [3.9848584845601014]
  We introduce the Cooperative Network Architecture (CNA), a model that represents sensory signals using structured, recurrently connected networks of neurons, termed "nets"<n>We demonstrate that net fragments can be learned without supervision and flexibly recombined to encode novel patterns, enabling figure completion and resilience to noise.
  arXiv  Detail & Related papers  (2024-07-08T06:22:10Z)
• Addressing caveats of neural persistence with deep graph persistence [54.424983583720675]
  We find that the variance of network weights and spatial concentration of large weights are the main factors that impact neural persistence. We propose an extension of the filtration underlying neural persistence to the whole neural network instead of single layers. This yields our deep graph persistence measure, which implicitly incorporates persistent paths through the network and alleviates variance-related issues.
  arXiv  Detail & Related papers  (2023-07-20T13:34:11Z)
• From NeurODEs to AutoencODEs: a mean-field control framework for width-varying Neural Networks [68.8204255655161]
  We propose a new type of continuous-time control system, called AutoencODE, based on a controlled field that drives dynamics. We show that many architectures can be recovered in regions where the loss function is locally convex.
  arXiv  Detail & Related papers  (2023-07-05T13:26:17Z)
• Intensity Profile Projection: A Framework for Continuous-Time Representation Learning for Dynamic Networks [50.2033914945157]
  We present a representation learning framework, Intensity Profile Projection, for continuous-time dynamic network data. The framework consists of three stages: estimating pairwise intensity functions, learning a projection which minimises a notion of intensity reconstruction error. Moreoever, we develop estimation theory providing tight control on the error of any estimated trajectory, indicating that the representations could even be used in quite noise-sensitive follow-on analyses.
  arXiv  Detail & Related papers  (2023-06-09T15:38:25Z)
• Latent Traversals in Generative Models as Potential Flows [113.4232528843775]
  We propose to model latent structures with a learned dynamic potential landscape. Inspired by physics, optimal transport, and neuroscience, these potential landscapes are learned as physically realistic partial differential equations. Our method achieves both more qualitatively and quantitatively disentangled trajectories than state-of-the-art baselines.
  arXiv  Detail & Related papers  (2023-04-25T15:53:45Z)
• Understanding Convolutional Neural Networks from Theoretical Perspective via Volterra Convolution [22.058311878382142]
  This study explores the relationship between convolutional neural networks and finite Volterra convolutions. It provides a novel approach to explain and study the overall characteristics of neural networks without being disturbed by the complex network architectures.
  arXiv  Detail & Related papers  (2021-10-19T12:07:46Z)
• Phase space learning with neural networks [0.0]
  This work proposes an autoencoder neural network as a non-linear generalization of projection-based methods for solving Partial Differential Equations (PDEs) The proposed deep learning architecture is capable of generating the dynamics of PDEs by integrating them completely in a very reduced latent space without intermediate reconstructions, to then decode the latent solution back to the original space. It is shown the reliability of properly regularized neural networks to learn the global characteristics of a dynamical system's phase space from the sample data of a single path, as well as its ability to predict unseen bifurcations.
  arXiv  Detail & Related papers  (2020-06-22T20:28:07Z)
• An Ode to an ODE [78.97367880223254]
  We present a new paradigm for Neural ODE algorithms, called ODEtoODE, where time-dependent parameters of the main flow evolve according to a matrix flow on the group O(d) This nested system of two flows provides stability and effectiveness of training and provably solves the gradient vanishing-explosion problem.
  arXiv  Detail & Related papers  (2020-06-19T22:05:19Z)
• Network Diffusions via Neural Mean-Field Dynamics [52.091487866968286]
  We propose a novel learning framework for inference and estimation problems of diffusion on networks. Our framework is derived from the Mori-Zwanzig formalism to obtain an exact evolution of the node infection probabilities. Our approach is versatile and robust to variations of the underlying diffusion network models.
  arXiv  Detail & Related papers  (2020-06-16T18:45:20Z)
• Mehler's Formula, Branching Process, and Compositional Kernels of Deep Neural Networks [3.167685495996986]
  We utilize a connection between compositional kernels and branching processes via Mehler's formula to study deep neural networks. We study the unscaled and rescaled limits of the compositional kernels and explore the different phases of the limiting behavior. Explicit formulas on the eigenvalues of the compositional kernel are provided, which quantify the complexity of the corresponding kernel Hilbert space.
  arXiv  Detail & Related papers  (2020-04-09T18:46:13Z)
• Local Propagation in Constraint-based Neural Network [77.37829055999238]
  We study a constraint-based representation of neural network architectures. We investigate a simple optimization procedure that is well suited to fulfil the so-called architectural constraints.
  arXiv  Detail & Related papers  (2020-02-18T16:47:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.