Spectral Saliency for Machine Unlearning

• URL: http://arxiv.org/abs/2608.15548v1
• Date: Sun, 16 Aug 2026 05:44:53 GMT
• Title: Spectral Saliency for Machine Unlearning
• Abstract summary: Machine unlearning (MU) aims to remove the influence of specific training data while preserving model utility.<n>We propose Spectral Saliency Unlearning (SSU) to thresholds weak singular components and updates only those directions supported by a confident unlearning signal.
• Score: 30.769424654889235
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine unlearning (MU) aims to remove the influence of specific training data while preserving model utility. As the name suggests, MU can be viewed as the inverse of learning, using gradient-based updates to reduce the influence of a forget-set by counteracting the previously learned behavior. Recently, Muon, a gradient descent variant, has been introduced. Muon applies spectral magnitude normalization to encourage exploration of rare directions and demonstrates promising performance. Inspired by Muon, we adopt the spectral view for unlearning and propose Spectral Saliency Unlearning (SSU). SSU thresholds weak singular components and updates only those directions supported by a confident unlearning signal. We further provide theoretical justification for this thresholding approach from the perspective of the forgetting-retention trade-off. Experiments across image classifiers, diffusion models, and LLMs demonstrate SSU's effectiveness.

Related papers

• The Spectral Dynamics and Noise Geometry of Muon [0.6335848702857036]
  Muon replaces a matrix gradient $G=UVtop$ by its polar factor $UVtop$.<n>We study the optimization bias created by this operation.
  arXiv  Detail & Related papers  (2026-06-07T00:51:30Z)
• Denoise First, Orthogonalize Later: Understanding Momentum in Muon via Spectral Filtering [32.15111956211244]
  We show that momentum in Muon acts as a spectral filter.<n>Applying momentum before perturbation provably stronger alignment with the signal component of gradient than either reversing this order or simply removing momentum.
  arXiv  Detail & Related papers  (2026-06-02T16:54:38Z)
• Uniform Diffusion Models Revisited: Leave-One-Out Denoiser and Absorbing State Reformulation [72.05592785529404]
  We show that the standard plug-in bridge parameterization for UDM is not optimized by the denoising posterior.<n>We also introduce an absorbing-state reformulation of uniform diffusion that preserves the UDM joint law while decomposing it into masked-diffusion-like sampling operations.
  arXiv  Detail & Related papers  (2026-05-21T17:27:19Z)
• DynMuon: A Dynamic Spectral Shaping View of Muon [12.175075916077702]
  We consider a class of Muon-like updates, where we replace the update $M$ with $Up Vtop$ for some parameter $p$.<n>We propose DynMuon, an efficient dynamic spectral shaping method that schedules $p$ from positive to mildly negative over training.
  arXiv  Detail & Related papers  (2026-05-16T18:30:11Z)
• SRL-MAD: Structured Residual Latents for One-Class Morphing Attack Detection [5.931399156681511]
  We introduce SRL-MAD, a one-class single-image MAD that uses structured residual Fourier representations for open-set morphing attack detection.<n>We show that SRL-MAD consistently outperforms recent one-class and supervised MAD models.
  arXiv  Detail & Related papers  (2026-03-16T10:05:44Z)
• Generative Model Inversion Through the Lens of the Manifold Hypothesis [98.37040155914595]
  Model inversion attacks (MIAs) aim to reconstruct class-representative samples from trained models.<n>Recent generative MIAs utilize generative adversarial networks to learn image priors that guide the inversion process.
  arXiv  Detail & Related papers  (2025-09-24T14:39:25Z)
• Frequency Domain-Based Diffusion Model for Unpaired Image Dehazing [92.61216319417208]
  We propose a novel frequency domain-based diffusion model, named ours, for fully exploiting the beneficial knowledge in unpaired clear data.<n>Inspired by the strong generative ability shown by Diffusion Models (DMs), we tackle the dehazing task from the perspective of frequency domain reconstruction.
  arXiv  Detail & Related papers  (2025-07-02T01:22:46Z)
• One-for-More: Continual Diffusion Model for Anomaly Detection [63.50488826645681]
  Anomaly detection methods utilize diffusion models to generate or reconstruct normal samples when given arbitrary anomaly images.<n>Our study found that the diffusion model suffers from severe faithfulness hallucination'' and catastrophic forgetting''<n>We propose a continual diffusion model that uses gradient projection to achieve stable continual learning.
  arXiv  Detail & Related papers  (2025-02-27T07:47:27Z)
• Understanding the Role of Equivariance in Self-supervised Learning [51.56331245499712]
  equivariant self-supervised learning (E-SSL) learns features to be augmentation-aware. We identify a critical explaining-away effect in E-SSL that creates a synergy between the equivariant and classification tasks. We reveal several principles for practical designs of E-SSL.
  arXiv  Detail & Related papers  (2024-11-10T16:09:47Z)
• Interpreting and Improving Diffusion Models from an Optimization Perspective [4.5993996573872185]
  We use this observation to interpret denoising diffusion models as approximate gradient descent applied to the Euclidean distance function. We propose a new gradient-estimation sampler, generalizing DDIM using insights from our theoretical results.
  arXiv  Detail & Related papers  (2023-06-08T00:56:33Z)
• SSMTL++: Revisiting Self-Supervised Multi-Task Learning for Video Anomaly Detection [108.57862846523858]
  We revisit the self-supervised multi-task learning framework, proposing several updates to the original method. We modernize the 3D convolutional backbone by introducing multi-head self-attention modules. In our attempt to further improve the model, we study additional self-supervised learning tasks, such as predicting segmentation maps.
  arXiv  Detail & Related papers  (2022-07-16T19:25:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.