A differentiable short-time Fourier transform with respect to the window length

• URL: http://arxiv.org/abs/2208.10886v2
• Date: Thu, 25 Aug 2022 08:57:10 GMT
• Title: A differentiable short-time Fourier transform with respect to the window length
• Authors: Maxime Leiber, Axel Barrau, Yosra Marnissi, Dany Abboud
• Abstract summary: We revisit the use of spectrograms in neural networks, by making the window length a continuous parameter optimizable by descent gradient. The contribution is mostly theoretical at this point, but plugging the modified STFT into any existing neural network is straightforward.
• Score: 4.0527583944137255
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In this paper, we revisit the use of spectrograms in neural networks, by making the window length a continuous parameter optimizable by gradient descent instead of an empirically tuned integer-valued hyperparameter. The contribution is mostly theoretical at this point, but plugging the modified STFT into any existing neural network is straightforward. We first define a differentiable version of the STFT in the case where local bins centers are fixed and independent of the window length parameter. We then discuss the more difficult case where the window length affects the position and number of bins. We illustrate the benefits of this new tool on an estimation and a classification problems, showing it can be of interest not only to neural networks but to any STFT-based signal processing algorithm.

Related papers

• Learnable Adaptive Time-Frequency Representation via Differentiable Short-Time Fourier Transform [11.05158127763157]
  We propose a differentiable formulation of the STFT that enables gradient-based optimization of its parameters.<n>Our approach integrates seamlessly with neural networks, allowing joint optimization of the STFT parameters and network weights.<n>The efficacy of the proposed differentiable STFT in enhancing TFRs and improving performance in downstream tasks is demonstrated.
  arXiv  Detail & Related papers  (2025-06-26T16:24:27Z)
• Random at First, Fast at Last: NTK-Guided Fourier Pre-Processing for Tabular DL [4.6774351030379835]
  We revisit and repurpose random Fourier mappings as a parameter-free, architecture-agnostic transformation.<n>We show that this approach circumvents the need for ad hoc normalization or additional learnable embeddings.<n> Empirically, we demonstrate that deep networks trained on Fourier-transformed inputs converge more rapidly and consistently achieve strong final performance.
  arXiv  Detail & Related papers  (2025-06-03T03:45:13Z)
• How Learnable Grids Recover Fine Detail in Low Dimensions: A Neural Tangent Kernel Analysis of Multigrid Parametric Encodings [106.3726679697804]
  We compare the two most common techniques for mitigating this spectral bias: Fourier feature encodings (FFE) and multigrid parametric encodings (MPE) MPEs are seen as the standard for low dimensional mappings, but MPEs often outperform them and learn representations with higher resolution and finer detail. We prove that MPEs improve a network's performance through the structure of their grid and not their learnable embedding.
  arXiv  Detail & Related papers  (2025-04-18T02:18:08Z)
• Neural Fields with Thermal Activations for Arbitrary-Scale Super-Resolution [56.089473862929886]
  We present a novel way to design neural fields such that points can be queried with an adaptive Gaussian PSF. With its theoretically guaranteed anti-aliasing, our method sets a new state of the art for arbitrary-scale single image super-resolution.
  arXiv  Detail & Related papers  (2023-11-29T14:01:28Z)
• Differentiable short-time Fourier transform with respect to the hop length [4.664495510551647]
  We propose a differentiable version of the short-time Fourier transform (STFT) that allows for gradient-based optimization of the hop length or the frame temporal position. Our approach provides improved control over the temporal positioning of frames, as the continuous nature of the hop length allows for a more finely-tuned optimization.
  arXiv  Detail & Related papers  (2023-07-26T07:04:09Z)
• Differentiable adaptive short-time Fourier transform with respect to the window length [4.664495510551647]
  This paper presents a gradient-based method for on-the-fly optimization for both per-frame and per-frequency window length of the short-time Fourier transform (STFT) The resulting differentiable adaptive STFT possesses commendable properties, such as the ability to adapt in the same time-frequency representation to both transient and stationary components, while being easily optimized by gradient descent.
  arXiv  Detail & Related papers  (2023-07-26T06:55:42Z)
• Efficient Parametric Approximations of Neural Network Function Space Distance [6.117371161379209]
  It is often useful to compactly summarize important properties of model parameters and training data so that they can be used later without storing and/or iterating over the entire dataset. We consider estimating the Function Space Distance (FSD) over a training set, i.e. the average discrepancy between the outputs of two neural networks. We propose a Linearized Activation TRick (LAFTR) and derive an efficient approximation to FSD for ReLU neural networks.
  arXiv  Detail & Related papers  (2023-02-07T15:09:23Z)
• Functional Regularization for Reinforcement Learning via Learned Fourier Features [98.90474131452588]
  We propose a simple architecture for deep reinforcement learning by embedding inputs into a learned Fourier basis. We show that it improves the sample efficiency of both state-based and image-based RL.
  arXiv  Detail & Related papers  (2021-12-06T18:59:52Z)
• Factorized Fourier Neural Operators [77.47313102926017]
  The Factorized Fourier Neural Operator (F-FNO) is a learning-based method for simulating partial differential equations. We show that our model maintains an error rate of 2% while still running an order of magnitude faster than a numerical solver.
  arXiv  Detail & Related papers  (2021-11-27T03:34:13Z)
• Rate Region for Indirect Multiterminal Source Coding in Federated Learning [49.574683687858126]
  A large number of edge devices send their updates to the edge at each round of the local model. Existing works do not leverage the focus in the information transmitted by different edges. This paper studies the rate region for the indirect multiterminal source coding FL.
  arXiv  Detail & Related papers  (2021-01-21T16:17:39Z)
• Optimizing Short-Time Fourier Transform Parameters via Gradient Descent [24.80575785857326]
  We show an approach that allows us to obtain a gradient for STFT parameters with respect to arbitrary cost functions. We do so for parameter values that stay constant throughout an input, but also for cases where these parameters have to dynamically change over time to accommodate varying signal characteristics.
  arXiv  Detail & Related papers  (2020-10-28T15:49:56Z)
• Efficient Long-Range Convolutions for Point Clouds [16.433511770049336]
  We present a novel neural network layer that directly incorporates long-range information for a point cloud. The LRC-layer is a particularly powerful tool when combined with local convolution. We showcase this framework by introducing a neural network architecture that combines LRC-layers with short-range convolutional layers.
  arXiv  Detail & Related papers  (2020-10-11T17:42:54Z)
• Network Adjustment: Channel Search Guided by FLOPs Utilization Ratio [101.84651388520584]
  This paper presents a new framework named network adjustment, which considers network accuracy as a function of FLOPs. Experiments on standard image classification datasets and a wide range of base networks demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2020-04-06T15:51:00Z)
• On the infinite width limit of neural networks with a standard parameterization [52.07828272324366]
  We propose an improved extrapolation of the standard parameterization that preserves all of these properties as width is taken to infinity. We show experimentally that the resulting kernels typically achieve similar accuracy to those resulting from an NTK parameterization.
  arXiv  Detail & Related papers  (2020-01-21T01:02:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.