FeRA: Frequency-Energy Constrained Routing for Effective Diffusion Adaptation Fine-Tuning

• URL: http://arxiv.org/abs/2511.17979v1
• Date: Sat, 22 Nov 2025 08:46:18 GMT
• Title: FeRA: Frequency-Energy Constrained Routing for Effective Diffusion Adaptation Fine-Tuning
• Authors: Bo Yin, Xiaobin Hu, Xingyu Zhou, Peng-Tao Jiang, Yue Liao, Junwei Zhu, Jiangning Zhang, Ying Tai, Chengjie Wang, Shuicheng Yan,
• Abstract summary: FeRA establishes a comprehensive frequency energy framework for effective diffusion adaptation fine tuning.<n>It integrates seamlessly with adapter based tuning schemes and generalizes well across diffusion backbones and resolutions.
• Score: 124.5975705334805
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion models have achieved remarkable success in generative modeling, yet how to effectively adapt large pretrained models to new tasks remains challenging. We revisit the reconstruction behavior of diffusion models during denoising to unveil the underlying frequency energy mechanism governing this process. Building upon this observation, we propose FeRA, a frequency driven fine tuning framework that aligns parameter updates with the intrinsic frequency energy progression of diffusion. FeRA establishes a comprehensive frequency energy framework for effective diffusion adaptation fine tuning, comprising three synergistic components: (i) a compact frequency energy indicator that characterizes the latent bandwise energy distribution, (ii) a soft frequency router that adaptively fuses multiple frequency specific adapter experts, and (iii) a frequency energy consistency regularization that stabilizes diffusion optimization and ensures coherent adaptation across bands. Routing operates in both training and inference, with inference time routing dynamically determined by the latent frequency energy. It integrates seamlessly with adapter based tuning schemes and generalizes well across diffusion backbones and resolutions. By aligning adaptation with the frequency energy mechanism, FeRA provides a simple, stable, and compatible paradigm for effective and robust diffusion model adaptation.

Related papers

• Spectral Regularization for Diffusion Models [14.919876123456747]
  We propose a loss-level spectral regularization framework that augments standard diffusion training with differentiable Fourier- and wavelet-domain losses.<n>Our approach is compatible with DDPM, DDIM, and EDM formulations and introduces negligible computational overhead.
  arXiv  Detail & Related papers  (2026-03-02T22:39:02Z)
• Iterative Inference-time Scaling with Adaptive Frequency Steering for Image Super-Resolution [75.3690742776891]
  We propose Iterative Diffusion Inference-Time Scaling with Adaptive Frequency Steering (IAFS)<n>IAFS addresses the challenge of balancing perceptual quality and structural fidelity by progressively refining the generated image through iterative correction of structural deviations.<n>Experiments show that IAFS effectively resolves the perception-fidelity conflict, yielding consistently improved perceptual detail and structural accuracy, and outperforming existing inference-time scaling methods.
  arXiv  Detail & Related papers  (2025-12-29T15:09:20Z)
• Neural Beamforming with Doppler-Aware Sparse Attention for High Mobility Environments [35.90167049594104]
  Deep learning-based beamforming offers an alternative with nonlinear mappings from channel state information to beamforming weights.<n> Transformer-based models are particularly effective due to their ability to model long-range dependencies across time and frequency.<n>Recent studies address this issue through sparse attention mechanisms that reduce complexity while maintaining, yet often employ patterns that disregard channel dynamics.
  arXiv  Detail & Related papers  (2025-11-05T16:51:26Z)
• SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation [62.14510717860079]
  We propose a Synergistic Diffusion-Autoregression paradigm that unifies the training efficiency of autoregressive models with the parallel inference capability of diffusion.<n>SDAR performs a lightweight paradigm conversion that transforms a well-trained autoregressive (AR) model into a blockwise diffusion model through brief, data-efficient adaptation.<n>Building on this insight, SDAR achieves efficient AR-to-diffusion conversion with minimal cost, preserving AR-level performance while enabling parallel generation.
  arXiv  Detail & Related papers  (2025-10-07T17:29:28Z)
• Frequency Regulation for Exposure Bias Mitigation in Diffusion Models [15.61368627978836]
  We make a key observation: the energy of predicted noisy samples in the reverse process continuously declines compared to perturbed samples in the forward process.<n>We introduce a dynamic frequency regulation mechanism utilizing wavelet transforms, which separately adjusts the low- and high-frequency subbands.<n>We derive the rigorous mathematical form of exposure bias.
  arXiv  Detail & Related papers  (2025-07-14T08:58:38Z)
• Broad Spectrum Coherent Frequency Conversion with Kinetic Inductance Superconducting Metastructures [7.527714650292084]
  Parametric frequency converters (PFCs) play a critical role in bridging the frequency gap between quantum information carriers.<n>Here, we present a multimode kinetic metastructure for parametric frequency conversion between broadly spanning frequency modes.
  arXiv  Detail & Related papers  (2025-03-12T04:42:38Z)
• Online Event-Triggered Switching for Frequency Control in Power Grids with Variable Inertia [5.344266377824671]
  Integration of renewable energy resources into power grids has led to time-varying system inertia and consequent degradation in frequency dynamics. A promising solution is using power electronics interfaced energy resources, such as renewable generators and battery energy storage for primary frequency control, by adjusting their power output set-points in response to frequency deviations. We model the frequency dynamics under time-varying inertia as a nonlinear switching system, where the frequency dynamics under each mode are described by the nonlinear swing equations and different modes represent different inertia levels. We present an online event-triggered switching algorithm to select the most suitable controller from a set of Neural
  arXiv  Detail & Related papers  (2024-08-27T22:44:33Z)
• Function Approximation for Reinforcement Learning Controller for Energy from Spread Waves [69.9104427437916]
  Multi-generator Wave Energy Converters (WEC) must handle multiple simultaneous waves coming from different directions called spread waves. These complex devices need controllers with multiple objectives of energy capture efficiency, reduction of structural stress to limit maintenance, and proactive protection against high waves. In this paper, we explore different function approximations for the policy and critic networks in modeling the sequential nature of the system dynamics.
  arXiv  Detail & Related papers  (2024-04-17T02:04:10Z)
• RF-Diffusion: Radio Signal Generation via Time-Frequency Diffusion [15.175370227353406]
  We introduce a novel Time-Frequency Diffusion theory to enhance the original diffusion model, enabling it to tap into the information within the time, frequency, and complex-valued domains of RF signals. RF-Diffusion is a versatile solution to generate diverse, high-quality, and time-series RF data. We also showcase the versatility of RF-Diffusion in boosting Wi-Fi sensing systems and performing channel estimation in 5G networks.
  arXiv  Detail & Related papers  (2024-04-14T04:56:05Z)
• Diffusion Probabilistic Model Made Slim [128.2227518929644]
  We introduce a customized design for slim diffusion probabilistic models (DPM) for light-weight image synthesis. We achieve 8-18x computational complexity reduction as compared to the latent diffusion models on a series of conditional and unconditional image generation tasks.
  arXiv  Detail & Related papers  (2022-11-27T16:27:28Z)
• Diffusion-GAN: Training GANs with Diffusion [135.24433011977874]
  Generative adversarial networks (GANs) are challenging to train stably. We propose Diffusion-GAN, a novel GAN framework that leverages a forward diffusion chain to generate instance noise. We show that Diffusion-GAN can produce more realistic images with higher stability and data efficiency than state-of-the-art GANs.
  arXiv  Detail & Related papers  (2022-06-05T20:45:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.