Harmonic Decompositions of Convolutional Networks

• URL: http://arxiv.org/abs/2003.12756v2
• Date: Mon, 16 Nov 2020 07:43:09 GMT
• Title: Harmonic Decompositions of Convolutional Networks
• Authors: Meyer Scetbon and Zaid Harchaoui
• Abstract summary: We show that the mapping associated with a convolutional network expands into a sum involving elementary functions akin to spherical harmonics. This functional decomposition can be related to the functional ANOVA decomposition in nonparametric statistics.
• Score: 16.688727673221297
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a description of the function space and the smoothness class associated with a convolutional network using the machinery of reproducing kernel Hilbert spaces. We show that the mapping associated with a convolutional network expands into a sum involving elementary functions akin to spherical harmonics. This functional decomposition can be related to the functional ANOVA decomposition in nonparametric statistics. Building off our functional characterization of convolutional networks, we obtain statistical bounds highlighting an interesting trade-off between the approximation error and the estimation error.

Related papers

• Distributed Learning with Discretely Observed Functional Data [1.4583059436979549]
  This paper combines distributed spectral algorithms with Sobolev kernels to tackle the functional linear regression problem. The hypothesis function spaces of the algorithms are the Sobolev spaces generated by the Sobolev kernels. We derive matching upper and lower bounds for the convergence of the distributed spectral algorithms in the Sobolev norm.
  arXiv  Detail & Related papers  (2024-10-03T10:49:34Z)
• Approximation Error and Complexity Bounds for ReLU Networks on Low-Regular Function Spaces [0.0]
  We consider the approximation of a large class of bounded functions, with minimal regularity assumptions, by ReLU neural networks. We show that the approximation error can be bounded from above by a quantity proportional to the uniform norm of the target function.
  arXiv  Detail & Related papers  (2024-05-10T14:31:58Z)
• Approximation of RKHS Functionals by Neural Networks [30.42446856477086]
  We study the approximation of functionals on kernel reproducing Hilbert spaces (RKHS's) using neural networks. We derive explicit error bounds for those induced by inverse multiquadric, Gaussian, and Sobolev kernels. We apply our findings to functional regression, proving that neural networks can accurately approximate the regression maps.
  arXiv  Detail & Related papers  (2024-03-18T18:58:23Z)
• Functional Linear Non-Gaussian Acyclic Model for Causal Discovery [7.303542369216906]
  We develop a framework to identify causal relationships in brain-effective connectivity tasks involving fMRI and EEG datasets. We establish theoretical guarantees of the identifiability of the causal relationship among non-Gaussian random vectors and even random functions in infinite-dimensional Hilbert spaces. For real data, we focus on analyzing the brain connectivity patterns derived from fMRI data.
  arXiv  Detail & Related papers  (2024-01-17T23:27:48Z)
• Moving mirror-field dynamics under intrinsic decoherence [77.34726150561087]
  We study the decaying dynamics in the mirror-field interaction by means of the intrinsic decoherence scheme. We show expectation values, correlations, and Husimi functions for the solutions obtained.
  arXiv  Detail & Related papers  (2023-05-06T03:41:45Z)
• On the renormalization group fixed point of the two-dimensional Ising model at criticality [77.34726150561087]
  We show that a simple, explicit analytic description of a fixed point using operator-algebraic renormalization (OAR) is possible. Specifically, the fixed point is characterized in terms of spin-spin correlation functions.
  arXiv  Detail & Related papers  (2023-04-06T16:57:28Z)
• Unified Fourier-based Kernel and Nonlinearity Design for Equivariant Networks on Homogeneous Spaces [52.424621227687894]
  We introduce a unified framework for group equivariant networks on homogeneous spaces. We take advantage of the sparsity of Fourier coefficients of the lifted feature fields. We show that other methods treating features as the Fourier coefficients in the stabilizer subgroup are special cases of our activation.
  arXiv  Detail & Related papers  (2022-06-16T17:59:01Z)
• Counting Phases and Faces Using Bayesian Thermodynamic Integration [77.34726150561087]
  We introduce a new approach to reconstruction of the thermodynamic functions and phase boundaries in two-parametric statistical mechanics systems. We use the proposed approach to accurately reconstruct the partition functions and phase diagrams of the Ising model and the exactly solvable non-equilibrium TASEP.
  arXiv  Detail & Related papers  (2022-05-18T17:11:23Z)
• Reinforcement Learning from Partial Observation: Linear Function Approximation with Provable Sample Efficiency [111.83670279016599]
  We study reinforcement learning for partially observed decision processes (POMDPs) with infinite observation and state spaces. We make the first attempt at partial observability and function approximation for a class of POMDPs with a linear structure.
  arXiv  Detail & Related papers  (2022-04-20T21:15:38Z)
• Space of Functions Computed by Deep-Layered Machines [74.13735716675987]
  We study the space of functions computed by random-layered machines, including deep neural networks and Boolean circuits. Investigating the distribution of Boolean functions computed on the recurrent and layer-dependent architectures, we find that it is the same in both models.
  arXiv  Detail & Related papers  (2020-04-19T18:31:03Z)
• Generalisation error in learning with random features and the hidden manifold model [23.71637173968353]
  We study generalised linear regression and classification for a synthetically generated dataset. We consider the high-dimensional regime and using the replica method from statistical physics. We show how to obtain the so-called double descent behaviour for logistic regression with a peak at the threshold. We discuss the role played by correlations in the data generated by the hidden manifold model.
  arXiv  Detail & Related papers  (2020-02-21T14:49:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.