LFA applied to CNNs: Efficient Singular Value Decomposition of Convolutional Mappings by Local Fourier Analysis

• URL: http://arxiv.org/abs/2506.05617v1
• Date: Thu, 05 Jun 2025 22:10:01 GMT
• Title: LFA applied to CNNs: Efficient Singular Value Decomposition of Convolutional Mappings by Local Fourier Analysis
• Authors: Antonia van Betteray, Matthias Rottmann, Karsten Kahl,
• Abstract summary: singular values of convolutional mappings encode interesting spectral properties.<n> computation of singular values is typically very resource-intensive.<n>We propose an approach of complexity O(N) based on local Fourier analysis.
• Score: 4.69726714177332
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The singular values of convolutional mappings encode interesting spectral properties, which can be used, e.g., to improve generalization and robustness of convolutional neural networks as well as to facilitate model compression. However, the computation of singular values is typically very resource-intensive. The naive approach involves unrolling the convolutional mapping along the input and channel dimensions into a large and sparse two-dimensional matrix, making the exact calculation of all singular values infeasible due to hardware limitations. In particular, this is true for matrices that represent convolutional mappings with large inputs and a high number of channels. Existing efficient methods leverage the Fast Fourier transformation (FFT) to transform convolutional mappings into the frequency domain, enabling the computation of singular values for matrices representing convolutions with larger input and channel dimensions. For a constant number of channels in a given convolution, an FFT can compute N singular values in O(N log N) complexity. In this work, we propose an approach of complexity O(N) based on local Fourier analysis, which additionally exploits the shift invariance of convolutional operators. We provide a theoretical analysis of our algorithm's runtime and validate its efficiency through numerical experiments. Our results demonstrate that our proposed method is scalable and offers a practical solution to calculate the entire set of singular values - along with the corresponding singular vectors if needed - for high-dimensional convolutional mappings.

Related papers

• Smooth Integer Encoding via Integral Balance [0.0]
  We introduce a novel method for encoding using smooth real-valued functions.<n>Our approach encodes the number N in the set of natural numbers through the cumulative balance of a smooth function f_N(t)<n>The total integral I(N) converges to zero as N tends to infinity, and the integer can be recovered as the minimal point of near-cancellation.
  arXiv  Detail & Related papers  (2025-04-28T20:23:53Z)
• Preconditioned Additive Gaussian Processes with Fourier Acceleration [2.292881746604941]
  We introduce a matrix-free method to achieve nearly linear complexity in the multiplication of kernel matrices and their derivatives.<n>To address high-dimensional problems, we propose an additive kernel approach.<n>Each sub- Kernel captures lower-order feature interactions, allowing for the efficient application of the NFFT method.
  arXiv  Detail & Related papers  (2025-04-01T07:14:06Z)
• Variable-size Symmetry-based Graph Fourier Transforms for image compression [65.7352685872625]
  We propose a new family of Symmetry-based Graph Fourier Transforms of variable sizes into a coding framework. Our proposed algorithm generates symmetric graphs on the grid by adding specific symmetrical connections between nodes. Experiments show that SBGFTs outperform the primary transforms integrated in the explicit Multiple Transform Selection.
  arXiv  Detail & Related papers  (2024-11-24T13:00:44Z)
• Quantum eigenvalue processing [0.0]
  Problems in linear algebra can be solved on a quantum computer by processing eigenvalues of the non-normal input matrices. We present a Quantum EigenValue Transformation (QEVT) framework for applying arbitrary transformations on eigenvalues of block-encoded non-normal operators. We also present a Quantum EigenValue Estimation (QEVE) algorithm for operators with real spectra.
  arXiv  Detail & Related papers  (2024-01-11T19:49:31Z)
• Beyond Regular Grids: Fourier-Based Neural Operators on Arbitrary Domains [13.56018270837999]
  We propose a simple method to extend neural operators to arbitrary domains. An efficient implementation* of such direct spectral evaluations is coupled with existing neural operator models. We demonstrate that the proposed method allows us to extend neural operators to arbitrary point distributions with significant gains in training speed over baselines.
  arXiv  Detail & Related papers  (2023-05-31T09:01:20Z)
• D4FT: A Deep Learning Approach to Kohn-Sham Density Functional Theory [79.50644650795012]
  We propose a deep learning approach to solve Kohn-Sham Density Functional Theory (KS-DFT) We prove that such an approach has the same expressivity as the SCF method, yet reduces the computational complexity. In addition, we show that our approach enables us to explore more complex neural-based wave functions.
  arXiv  Detail & Related papers  (2023-03-01T10:38:10Z)
• Transform Once: Efficient Operator Learning in Frequency Domain [69.74509540521397]
  We study deep neural networks designed to harness the structure in frequency domain for efficient learning of long-range correlations in space or time. This work introduces a blueprint for frequency domain learning through a single transform: transform once (T1)
  arXiv  Detail & Related papers  (2022-11-26T01:56:05Z)
• Nonparametric Factor Trajectory Learning for Dynamic Tensor Decomposition [20.55025648415664]
  We propose NON FActor Trajectory learning for dynamic tensor decomposition (NONFAT) We use a second-level GP to sample the entry values and to capture the temporal relationship between the entities. We have shown the advantage of our method in several real-world applications.
  arXiv  Detail & Related papers  (2022-07-06T05:33:00Z)
• Factorized Fourier Neural Operators [77.47313102926017]
  The Factorized Fourier Neural Operator (F-FNO) is a learning-based method for simulating partial differential equations. We show that our model maintains an error rate of 2% while still running an order of magnitude faster than a numerical solver.
  arXiv  Detail & Related papers  (2021-11-27T03:34:13Z)
• Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformers [55.90468016961356]
  We propose an efficient token mixer that learns to mix in the Fourier domain. AFNO is based on a principled foundation of operator learning. It can handle a sequence size of 65k and outperforms other efficient self-attention mechanisms.
  arXiv  Detail & Related papers  (2021-11-24T05:44:31Z)
• Channel Assignment in Uplink Wireless Communication using Machine Learning Approach [54.012791474906514]
  This letter investigates a channel assignment problem in uplink wireless communication systems. Our goal is to maximize the sum rate of all users subject to integer channel assignment constraints. Due to high computational complexity, machine learning approaches are employed to obtain computational efficient solutions.
  arXiv  Detail & Related papers  (2020-01-12T15:54:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.