Music Restoration via Latent Operator Optimization and Diffusion Model Priors

• URL: http://arxiv.org/abs/2608.01972v1
• Date: Mon, 03 Aug 2026 09:36:40 GMT
• Title: Music Restoration via Latent Operator Optimization and Diffusion Model Priors
• Abstract summary: Music restoration seeks to recover a clean signal from an observed recording degraded by an unknown effect, distortion, or corruption.<n>We propose LOUDAR, a general-purpose restoration method that operates in the latent space of a pretrained audio autoencoder.<n>We show that it consistently improves over degraded inputs and is competitive with supervised and unsupervised baselines in waveform and latent domains.
• Score: 15.43777154891635
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Music restoration seeks to recover a clean signal from an observed recording degraded by an unknown effect, distortion, or corruption. Existing systems often rely on paired training data and distortion-specific supervision, which limits their use when the forward process is not known in advance. We propose LOUDAR (Latent-space Optimization of Unknown Distortion for Audio Restoration) a general-purpose restoration method that operates in the latent space of a pretrained audio autoencoder and models the unknown distortion as a learnable latent operator. At inference time, LOUDAR alternates between estimating the clean latent variable and updating the latent operator parameters. An unconditional latent diffusion model provides a prior over clean audio and regularizes this inference by steering the latent estimate toward the manifold of clean recordings. Because the degradation model is adapted per input, the approach is broadly applicable across diverse restoration problems. We evaluate LOUDAR on singing voice effect removal and restoration, as well as guitar distortion removal, and show that it consistently improves over degraded inputs and is competitive with supervised and unsupervised baselines in waveform and latent domains.

Related papers

• Vorch-Director: Interactive World Story Model via Noise-Aware Error Rectification [28.964205712747255]
  Vorch-Director is a noise-level-aware residual correction strategy.<n>It associates each residual with its originating noise level and injects residuals from matched noise regimes during training.<n>It produces more realistic autoregressive histories while retaining efficient teacher-forcing training.
  arXiv  Detail & Related papers  (2026-08-06T09:09:49Z)
• Next-Acceleration-Scale Prediction for Autoregressive MRI Reconstruction [52.32112533846212]
  MRI reconstruction is an inherently ill-posed inverse problem, since incomplete measurements admit many plausible solutions.<n>We address this limitation by moving reconstruction to discrete multi-scale latent space and posing it as autoregressive next-acceleration-scale prediction.<n>We show that our approach delivers improved reconstruction performance across diverse sampling patterns under extreme undersampling.
  arXiv  Detail & Related papers  (2026-05-19T04:40:50Z)
• Your Pre-trained Diffusion Model Secretly Knows Restoration [55.7186754179308]
  We show that pre-trained diffusion models inherently possess restoration behavior, which can be unlocked by directly learning prompt embeddings.<n>We introduce our lightweight learned prompts on the pre-trained WAN video model and FLUX image models, converting them into high-performing restoration models.
  arXiv  Detail & Related papers  (2026-04-06T17:59:04Z)
• Conditional Flow Matching for Visually-Guided Acoustic Highlighting [10.241097929338366]
  Visually-guided acoustic highlighting seeks to rebalance audio in alignment with the accompanying video, creating a coherent audio-visual experience.<n>Existing approaches use discriminative models, which struggle with the inherent ambiguity in audio remixing.<n>We introduce a Conditional Flow Matching (CFM) framework to address this limitation.
  arXiv  Detail & Related papers  (2026-02-03T17:24:47Z)
• CAPTAIN: Semantic Feature Injection for Memorization Mitigation in Text-to-Image Diffusion Models [60.610268549138375]
  Diffusion models can unintentionally reproduce training examples, raising privacy and copyright concerns.<n>We introduce CAPTAIN, a training-free framework that mitigates memorization by directly modifying latent features during denoising.
  arXiv  Detail & Related papers  (2025-12-11T14:01:47Z)
• Latent Harmony: Synergistic Unified UHD Image Restoration via Latent Space Regularization and Controllable Refinement [89.99237142387655]
  We introduce LH-VAE, which enhances semantic robustness through visual semantic constraints and progressive degradations.<n>Latent Harmony is a two-stage framework that redefines VAEs for UHD restoration by jointly regularizing the latent space and enforcing high-frequency-aware reconstruction.<n>Experiments show Latent Harmony achieves state-of-the-art performance across UHD and standard-resolution tasks, effectively balancing efficiency, perceptual quality, and reconstruction accuracy.
  arXiv  Detail & Related papers  (2025-10-09T08:54:26Z)
• Denoising as Adaptation: Noise-Space Domain Adaptation for Image Restoration [64.84134880709625]
  We show that it is possible to perform domain adaptation via the noise space using diffusion models.<n>In particular, by leveraging the unique property of how auxiliary conditional inputs influence the multi-step denoising process, we derive a meaningful diffusion loss.<n>We present crucial strategies such as channel-shuffling layer and residual-swapping contrastive learning in the diffusion model.
  arXiv  Detail & Related papers  (2024-06-26T17:40:30Z)
• Blind Audio Bandwidth Extension: A Diffusion-Based Zero-Shot Approach [4.030910640265943]
  In cases where the lowpass degradation is unknown, such as in restoring historical audio recordings, this becomes a blind problem. This paper introduces a novel method called BABE that addresses the blind problem in a zero-shot setting. BABE exhibits robust generalization capabilities when enhancing real historical recordings.
  arXiv  Detail & Related papers  (2023-06-02T10:47:15Z)
• DiffusionAD: Norm-guided One-step Denoising Diffusion for Anomaly Detection [80.20339155618612]
  DiffusionAD is a novel anomaly detection pipeline comprising a reconstruction sub-network and a segmentation sub-network.<n>A rapid one-step denoising paradigm achieves hundreds of times acceleration while preserving comparable reconstruction quality.<n>Considering the diversity in the manifestation of anomalies, we propose a norm-guided paradigm to integrate the benefits of multiple noise scales.
  arXiv  Detail & Related papers  (2023-03-15T16:14:06Z)
• Unsupervised vocal dereverberation with diffusion-based generative models [12.713895991763867]
  We propose an unsupervised method to remove a general kind of artificial reverb for music without requiring pairs of data for training. We show through objective and perceptual evaluations that our method outperforms the current leading vocal dereverberation benchmarks.
  arXiv  Detail & Related papers  (2022-11-08T09:43:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.