Optimizing Short-Time Fourier Transform Parameters via Gradient Descent

• URL: http://arxiv.org/abs/2010.15049v2
• Date: Thu, 18 Feb 2021 22:42:52 GMT
• Title: Optimizing Short-Time Fourier Transform Parameters via Gradient Descent
• Authors: An Zhao, Krishna Subramani and Paris Smaragdis
• Abstract summary: We show an approach that allows us to obtain a gradient for STFT parameters with respect to arbitrary cost functions. We do so for parameter values that stay constant throughout an input, but also for cases where these parameters have to dynamically change over time to accommodate varying signal characteristics.
• Score: 24.80575785857326
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The Short-Time Fourier Transform (STFT) has been a staple of signal processing, often being the first step for many audio tasks. A very familiar process when using the STFT is the search for the best STFT parameters, as they often have significant side effects if chosen poorly. These parameters are often defined in terms of an integer number of samples, which makes their optimization non-trivial. In this paper we show an approach that allows us to obtain a gradient for STFT parameters with respect to arbitrary cost functions, and thus enable the ability to employ gradient descent optimization of quantities like the STFT window length, or the STFT hop size. We do so for parameter values that stay constant throughout an input, but also for cases where these parameters have to dynamically change over time to accommodate varying signal characteristics.

Related papers

• Learnable Adaptive Time-Frequency Representation via Differentiable Short-Time Fourier Transform [11.05158127763157]
  We propose a differentiable formulation of the STFT that enables gradient-based optimization of its parameters.<n>Our approach integrates seamlessly with neural networks, allowing joint optimization of the STFT parameters and network weights.<n>The efficacy of the proposed differentiable STFT in enhancing TFRs and improving performance in downstream tasks is demonstrated.
  arXiv  Detail & Related papers  (2025-06-26T16:24:27Z)
• Visual Fourier Prompt Tuning [63.66866445034855]
  We propose the Visual Fourier Prompt Tuning (VFPT) method as a general and effective solution for adapting large-scale transformer-based models. Our approach incorporates the Fast Fourier Transform into prompt embeddings and harmoniously considers both spatial and frequency domain information. Our results demonstrate that our approach outperforms current state-of-the-art baselines on two benchmarks.
  arXiv  Detail & Related papers  (2024-11-02T18:18:35Z)
• Zeroth-Order Fine-Tuning of LLMs with Extreme Sparsity [66.67596152389591]
  Zeroth-order optimization (ZO) is a memory-efficient strategy for fine-tuning Large Language Models. In this study, we investigate the feasibility of fine-tuning an extremely small subset of LLM parameters using ZO. Our results demonstrate that fine-tuning 0.1% sensitive parameters in the LLM with ZO can outperform the full ZO fine-tuning performance.
  arXiv  Detail & Related papers  (2024-06-05T04:07:35Z)
• Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization [102.92240148504774]
  We study a principled finetuning paradigm -- Orthogonal Finetuning (OFT) -- for downstream task adaptation. Despite demonstrating good generalizability, OFT still uses a fairly large number of trainable parameters. We apply this parameterization to OFT, creating a novel parameter-efficient finetuning method, called Orthogonal Butterfly (BOFT)
  arXiv  Detail & Related papers  (2023-11-10T18:59:54Z)
• DePT: Decomposed Prompt Tuning for Parameter-Efficient Fine-tuning [14.975436239088312]
  We propose DePT, which decomposes the soft prompt into a shorter soft prompt and a pair of low-rank matrices that are then optimised with two different learning rates. We demonstrate that DePT outperforms state-of-the-art PEFT approaches, including the full fine-tuning baseline, in some scenarios.
  arXiv  Detail & Related papers  (2023-09-11T00:02:05Z)
• Differentiable short-time Fourier transform with respect to the hop length [4.664495510551647]
  We propose a differentiable version of the short-time Fourier transform (STFT) that allows for gradient-based optimization of the hop length or the frame temporal position. Our approach provides improved control over the temporal positioning of frames, as the continuous nature of the hop length allows for a more finely-tuned optimization.
  arXiv  Detail & Related papers  (2023-07-26T07:04:09Z)
• Differentiable adaptive short-time Fourier transform with respect to the window length [4.664495510551647]
  This paper presents a gradient-based method for on-the-fly optimization for both per-frame and per-frequency window length of the short-time Fourier transform (STFT) The resulting differentiable adaptive STFT possesses commendable properties, such as the ability to adapt in the same time-frequency representation to both transient and stationary components, while being easily optimized by gradient descent.
  arXiv  Detail & Related papers  (2023-07-26T06:55:42Z)
• Sensitivity-Aware Visual Parameter-Efficient Fine-Tuning [91.5113227694443]
  We propose a novel visual. sensuous-aware fine-Tuning (SPT) scheme. SPT allocates trainable parameters to task-specific important positions. Experiments on a wide range of downstream recognition tasks show that our SPT is complementary to the existing PEFT methods.
  arXiv  Detail & Related papers  (2023-03-15T12:34:24Z)
• Optimization using Parallel Gradient Evaluations on Multiple Parameters [51.64614793990665]
  We propose a first-order method for convex optimization, where gradients from multiple parameters can be used during each step of gradient descent. Our method uses gradients from multiple parameters in synergy to update these parameters together towards the optima.
  arXiv  Detail & Related papers  (2023-02-06T23:39:13Z)
• AutoPEFT: Automatic Configuration Search for Parameter-Efficient Fine-Tuning [77.61565726647784]
  Motivated by advances in neural architecture search, we propose AutoPEFT for automatic PEFT configuration selection. We show that AutoPEFT-discovered configurations significantly outperform existing PEFT methods and are on par or better than FFT without incurring substantial training efficiency costs.
  arXiv  Detail & Related papers  (2023-01-28T08:51:23Z)
• A differentiable short-time Fourier transform with respect to the window length [4.0527583944137255]
  We revisit the use of spectrograms in neural networks, by making the window length a continuous parameter optimizable by descent gradient. The contribution is mostly theoretical at this point, but plugging the modified STFT into any existing neural network is straightforward.
  arXiv  Detail & Related papers  (2022-08-23T11:38:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.