Clean2FX: Label-conditioned modeling for clean-to-effect guitar audio transformations

• URL: http://arxiv.org/abs/2607.08863v1
• Date: Thu, 09 Jul 2026 18:35:01 GMT
• Title: Clean2FX: Label-conditioned modeling for clean-to-effect guitar audio transformations
• Abstract summary: We present Clean2FX, a study and demo of label-conditioned clean-to-effect transformation for electric guitar audio.<n>Given a clean guitar input and a target effect label, the task is to synthesize the corresponding effected signal while preserving the musical content.<n>We evaluate four neural approaches under a common spectrogram-based transformation setting.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present Clean2FX, a study and demo of label-conditioned clean-to-effect transformation for electric guitar audio. Given a clean guitar input and a target effect label, the task is to synthesize the corresponding effected signal while preserving the musical content. Training and evaluation pairs are constructed from EGFxSet real, single tone recordings by assembling matched clean/effected chords, melodies, and mixed timelines. This allows for controlled comparison across effects. We evaluate four neural approaches under a common spectrogram-based transformation setting: two variational autoencoders and two U-Net models that differ in whether they operate on linear or log-magnitude representations. Performance is measured using linear-magnitude spectrogram MSE and Fréchet Audio Distance. The U-Net models outperform the variational autoencoder variants. Per-effect results show that distortion effects are most readily improved, whereas delay and reverb effects exhibit weaker FAD gains despite substantial spectral-error reductions. A conditioning-sensitivity diagnostic provides evidence that the best model responds to target labels rather than collapsing to a single transformation. Our demo website compares two models applied on real-world guitar performances outside training and validation data, providing audio and spectrogram examples of the practical clean-to-effect behavior.

Related papers

• TimberAgent: Gram-Guided Retrieval for Executable Music Effect Control [9.319159431126453]
  Texture Resonance Retrieval (TRR) is an audio representation built from Gram matrices of projected mid-level Wav2Vec2 activations.<n>We evaluate TRR on a guitar-effects benchmark with 1,063 candidate presets and 204 queries.
  arXiv  Detail & Related papers  (2026-03-10T08:09:32Z)
• Representation-Regularized Convolutional Audio Transformer for Audio Understanding [53.092757178419355]
  bootstrapping representations from scratch is computationally expensive, often requiring extensive training to converge.<n>We propose the Convolutional Audio Transformer (CAT), a unified framework designed to address these challenges.
  arXiv  Detail & Related papers  (2026-01-29T12:16:19Z)
• Gradient-based Optimisation of Modulation Effects [8.97214437002284]
  We present a framework for modelling flanger, chorus and phaser effects based on differentiable digital signal processing.<n>The model is trained in the time-frequency domain, but at inference operates in the time-domain, requiring zero latency.<n>We show that when trained against analog effects units, sound output from the model is in some cases perceptually indistinguishable from the reference.
  arXiv  Detail & Related papers  (2026-01-08T12:04:41Z)
• Open-Amp: Synthetic Data Framework for Audio Effect Foundation Models [4.569691863088947]
  This paper introduces Open-Amp, a synthetic data framework for generating large-scale and diverse audio effects data. Our experiments show that using Open-Amp to train a guitar effects encoder achieves new state-of-the-art results on multiple guitar effects classification tasks.
  arXiv  Detail & Related papers  (2024-11-22T14:27:59Z)
• DITTO: Diffusion Inference-Time T-Optimization for Music Generation [49.90109850026932]
  Diffusion Inference-Time T-Optimization (DITTO) is a frame-work for controlling pre-trained text-to-music diffusion models at inference-time. We demonstrate a surprisingly wide-range of applications for music generation including inpainting, outpainting, and looping as well as intensity, melody, and musical structure control.
  arXiv  Detail & Related papers  (2024-01-22T18:10:10Z)
• Towards Improving Harmonic Sensitivity and Prediction Stability for Singing Melody Extraction [36.45127093978295]
  We propose an input feature modification and a training objective modification based on two assumptions. To enhance the model's sensitivity on the trailing harmonics, we modify the Combined Frequency and Periodicity representation using discrete z-transform. We apply these modifications to several models, including MSNet, FTANet, and a newly introduced model, PianoNet, modified from a piano transcription network.
  arXiv  Detail & Related papers  (2023-08-04T21:59:40Z)
• From Discrete Tokens to High-Fidelity Audio Using Multi-Band Diffusion [84.138804145918]
  Deep generative models can generate high-fidelity audio conditioned on various types of representations. These models are prone to generate audible artifacts when the conditioning is flawed or imperfect. We propose a high-fidelity multi-band diffusion-based framework that generates any type of audio modality from low-bitrate discrete representations.
  arXiv  Detail & Related papers  (2023-08-02T22:14:29Z)
• Anomalous Sound Detection using Audio Representation with Machine ID based Contrastive Learning Pretraining [52.191658157204856]
  This paper uses contrastive learning to refine audio representations for each machine ID, rather than for each audio sample. The proposed two-stage method uses contrastive learning to pretrain the audio representation model. Experiments show that our method outperforms the state-of-the-art methods using contrastive learning or self-supervised classification.
  arXiv  Detail & Related papers  (2023-04-07T11:08:31Z)
• Removing Distortion Effects in Music Using Deep Neural Networks [12.497836634060569]
  This paper focuses on removing distortion and clipping applied to guitar tracks for music production. It presents a comparative investigation of different deep neural network (DNN) architectures on this task. We achieve exceptionally good results in distortion removal using DNNs for effects that superimpose the clean signal to the distorted signal.
  arXiv  Detail & Related papers  (2022-02-03T16:26:29Z)
• DiffSinger: Diffusion Acoustic Model for Singing Voice Synthesis [53.19363127760314]
  DiffSinger is a parameterized Markov chain which iteratively converts the noise into mel-spectrogram conditioned on the music score. The evaluations conducted on the Chinese singing dataset demonstrate that DiffSinger outperforms state-of-the-art SVS work with a notable margin.
  arXiv  Detail & Related papers  (2021-05-06T05:21:42Z)
• Real Time Speech Enhancement in the Waveform Domain [99.02180506016721]
  We present a causal speech enhancement model working on the raw waveform that runs in real-time on a laptop CPU. The proposed model is based on an encoder-decoder architecture with skip-connections. It is capable of removing various kinds of background noise including stationary and non-stationary noises.
  arXiv  Detail & Related papers  (2020-06-23T09:19:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.