FIBER: A Differentially Private Optimizer with Filter-Aware Innovation Bias Correction

• URL: http://arxiv.org/abs/2605.03425v1
• Date: Tue, 05 May 2026 07:02:32 GMT
• Title: FIBER: A Differentially Private Optimizer with Filter-Aware Innovation Bias Correction
• Abstract summary: We propose FiBeR, a DP designed for temporally filtered privatized gradients.<n>Across vision and language benchmarks, FiBeR consistently demonstrates substantial improvements in the performance of DPs.
• Score: 10.531164867030272
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Differentially private (DP) training protects individual examples by adding noise to gradients, but the injected noise interacts nontrivially with adaptive optimizers. Recent DP methods temporally filter privatized gradients to reduce variance; however, filtering also changes the DP noise statistics seen by AdamW's second-moment accumulator. As a result, bias corrections derived for unfiltered DP noise, such as subtracting sigma_w squared, can become miscalibrated when filtering is present. We propose FiBeR, a DP optimizer designed for temporally filtered privatized gradients. FiBeR (i) performs denoising in innovation space by filtering the residual stream and integrating it to form the filtered gradient estimate, (ii) decouples the two-point observation geometry from the innovation gain to enable independent tuning, and (iii) introduces a filter-aware second-moment calibration that subtracts the attenuated DP noise contribution A(omega) sigma_w squared, where A(omega) is derived in closed form for the innovation filter and can be computed for general stable linear filters. Across vision and language benchmarks, FiBeR consistently demonstrates substantial improvements in the performance of DP optimizers, surpassing state-of-the-art results under equivalent privacy constraints on multiple tasks.

Related papers

• Closed-Form Spectral Regularization for Multi-Task Model Merging [96.82449201305234]
  Model merging combines several independently fine-tuned experts into a single multi-task model without any training data.<n>State-of-the-art merging methods formulate merging as a layer-wise interference problem.<n>We propose SWUDI, a closed-form method that combines a soft exponential filter, which matches the gradient-flow trajectory of iterative descent.
  arXiv  Detail & Related papers  (2026-06-05T14:00:47Z)
• Deformable Wiener Filter for Future Video Coding [55.405068056883614]
  We propose a deformable Wiener Filter (DWF) for in-loop filters in Versatile Video Coding (VVC)<n>It combines the local and non-local characteristics and supervisedly trains the filter coefficients based on the Wiener Filter theory.<n>It achieves 1.16%, 1.92%, and 2.67% bit-rate savings on average compared to the VTM-11.0 for All Intra, Random Access, and Low-Delay B configurations.
  arXiv  Detail & Related papers  (2026-06-01T02:24:07Z)
• Differentiable Attenuation Filters for Feedback Delay Networks [3.8530395083350615]
  We introduce a novel method for designing attenuation filters in digital audio reverberation systems based on Feedback Delay Net- works (FDNs)<n>Our approach uses Second Order Sections (SOS) of Infinite Impulse Response (IIR) filters arranged as parametric equalizers (PEQ)<n>Our method delivers a flexible and differentiable design, achieving state-of-the-art per- formance while significantly reducing computational cost.
  arXiv  Detail & Related papers  (2025-11-25T15:01:55Z)
• Enhancing DPSGD via Per-Sample Momentum and Low-Pass Filtering [4.9871580445771455]
  Differentially Private Gradient Descent (DPSGD) is widely used to train deep neural networks with formal privacy guarantees.<n>Existing techniques typically address only one of these issues, as reducing DP noise can exacerbate clipping bias and vice-versa.<n>We propose a novel method, emphDP-PMLF, which integrates per-sample momentum with a low-pass filtering strategy to simultaneously mitigate DP noise and clipping bias.
  arXiv  Detail & Related papers  (2025-11-11T23:40:02Z)
• GSPRec: Temporal-Aware Graph Spectral Filtering for Recommendation [19.275813818441506]
  We introduce GSPRec, a graph spectral model that integrates temporal transitions through sequentially-informed graph construction.<n> GSPRec encodes item transitions via multi-hop diffusion to enable the use of symmetric Laplacians for spectral processing.<n>To capture user preferences, we design a dual-filtering mechanism: a Gaussian bandpass filter to extract mid-frequency, user-level patterns, and a low-pass filter to retain global trends.
  arXiv  Detail & Related papers  (2025-05-15T15:49:56Z)
• Fast Fourier Transform-Based Spectral and Temporal Gradient Filtering for Differential Privacy [1.5766082819987293]
  We introduce the FFT-Enhanced Kalman Filter (FFTKF), a differentially private optimization method that improves gradient quality.<n>FFTKF applies frequency-domain filtering to shift privacy noise into less informative high-frequency components.<n>It achieves higher test accuracy than DP-SGD and DiSK on MNIST, CIFAR-10, CIFAR-100, and Tiny-ImageNet with CNNs, Wide ResNets, and Vision Transformers.
  arXiv  Detail & Related papers  (2025-05-07T14:38:58Z)
• DiSK: Differentially Private Optimizer with Simplified Kalman Filter for Noise Reduction [57.83978915843095]
  This paper introduces DiSK, a novel framework designed to significantly enhance the performance of differentially private gradients.<n>To ensure practicality for large-scale training, we simplify the Kalman filtering process, minimizing its memory and computational demands.
  arXiv  Detail & Related papers  (2024-10-04T19:30:39Z)
• DOPPLER: Differentially Private Optimizers with Low-pass Filter for Privacy Noise Reduction [47.65999101635902]
  Differentially private (DP) training prevents the leakage of sensitive information in the collected training data from trained machine learning models. We develop a new component, called DOPPLER, which works by effectively amplifying the gradient while DP noise within this frequency domain. Our experiments show that the proposed DPs with a lowpass filter outperform their counterparts without the filter by 3%-10% in test accuracy.
  arXiv  Detail & Related papers  (2024-08-24T04:27:07Z)
• Closed-form Filtering for Non-linear Systems [83.91296397912218]
  We propose a new class of filters based on Gaussian PSD Models, which offer several advantages in terms of density approximation and computational efficiency. We show that filtering can be efficiently performed in closed form when transitions and observations are Gaussian PSD Models. Our proposed estimator enjoys strong theoretical guarantees, with estimation error that depends on the quality of the approximation and is adaptive to the regularity of the transition probabilities.
  arXiv  Detail & Related papers  (2024-02-15T08:51:49Z)
• Differentially Private SGD Without Clipping Bias: An Error-Feedback Approach [62.000948039914135]
  Using Differentially Private Gradient Descent with Gradient Clipping (DPSGD-GC) to ensure Differential Privacy (DP) comes at the cost of model performance degradation. We propose a new error-feedback (EF) DP algorithm as an alternative to DPSGD-GC. We establish an algorithm-specific DP analysis for our proposed algorithm, providing privacy guarantees based on R'enyi DP.
  arXiv  Detail & Related papers  (2023-11-24T17:56:44Z)
• Neural Network-augmented Kalman Filtering for Robust Online Speech Dereverberation in Noisy Reverberant Environments [13.49645012479288]
  A neural network-augmented algorithm for noise-robust online dereverberation is proposed. The presented framework allows for robust dereverberation on a single-channel noisy reverberant dataset.
  arXiv  Detail & Related papers  (2022-04-06T11:38:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.