Dynamic Frequency Modulation for Controllable Text-driven Image Generation

• URL: http://arxiv.org/abs/2602.10662v1
• Date: Wed, 11 Feb 2026 09:06:44 GMT
• Title: Dynamic Frequency Modulation for Controllable Text-driven Image Generation
• Abstract summary: modifying the original text prompt to achieve the expected semantic adjustments often results in unintended global structure changes.<n>This paper analyzes the impact of the frequency spectrum of noisy latent variables on the hierarchical emergence of the structure framework.<n>We propose a training-free frequency modulation method utilizing a frequency-dependent weighting function with dynamic decay.
• Score: 26.61127727590205
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The success of text-guided diffusion models has established a new image generation paradigm driven by the iterative refinement of text prompts. However, modifying the original text prompt to achieve the expected semantic adjustments often results in unintended global structure changes that disrupt user intent. Existing methods rely on empirical feature map selection for intervention, whose performance heavily depends on appropriate selection, leading to suboptimal stability. This paper tries to solve the aforementioned problem from a frequency perspective and analyzes the impact of the frequency spectrum of noisy latent variables on the hierarchical emergence of the structure framework and fine-grained textures during the generation process. We find that lower-frequency components are primarily responsible for establishing the structure framework in the early generation stage. Their influence diminishes over time, giving way to higher-frequency components that synthesize fine-grained textures. In light of this, we propose a training-free frequency modulation method utilizing a frequency-dependent weighting function with dynamic decay. This method maintains the structure framework consistency while permitting targeted semantic modifications. By directly manipulating the noisy latent variable, the proposed method avoids the empirical selection of internal feature maps. Extensive experiments demonstrate that the proposed method significantly outperforms current state-of-the-art methods, achieving an effective balance between preserving structure and enabling semantic updates.

Related papers

• In-Context Forcing: Uncovering Context Effects in Autoregressive Video Diffusion [52.18702395026718]
  We introduce In-Context Forcing, a progressive autoregressive paradigm that utilizes contexts with decreasing noise levels.<n>By applying less masking to distant frames and more masking to adjacent ones, this approach provides adaptive guidance.<n>Our method significantly outperforms state-of-the-art approaches in both visual fidelity and inference speed.
  arXiv  Detail & Related papers  (2026-08-05T13:54:31Z)
• Controllable Histopathology Image Synthesis with Training-free Structural Initialization and Textural Modulation [8.02064374903123]
  We present CHIS, a plug-in framework that guides the sampling trajectory of a pretrained diffusion model through two key stages.<n>During the reverse diffusion process, we adaptively modulate both coarse-grained and fine-grained textures at different wavelet decomposition levels.<n>This enables a diffusion model pretrained solely on unlabeled images to generate outputs that align with prior structural masks.
  arXiv  Detail & Related papers  (2026-06-26T10:27:03Z)
• Temporally-Aligned Evaluation for Audio-Driven Talking Head Generation [28.732587811107777]
  Existing evaluation protocols mainly rely on frame-wise metrics that assume strict temporal correspondence between generated and reference videos.<n>We introduce a unified sequence-level reformulation that integrates Soft Dynamic Time Warping into established evaluation pipelines.<n>We show that frame-wise evaluation can be viewed as a special case under rigid alignment, while sequence-level alignment provides improved stability, lower sensitivity to timing differences, and clearer separation between modeling paradigms.
  arXiv  Detail & Related papers  (2026-05-31T05:44:42Z)
• InpaintSLat: Inpainting Structured 3D Latents via Initial Noise Optimization [56.525920531932606]
  We present a training-free approach for controllable 3D inpainting based on initial noise optimization.<n>In this paper, we introduce a strategy to optimize the initial noise within the structured 3D latent diffusion framework.
  arXiv  Detail & Related papers  (2026-05-01T13:45:25Z)
• Iterative Inference-time Scaling with Adaptive Frequency Steering for Image Super-Resolution [75.3690742776891]
  We propose Iterative Diffusion Inference-Time Scaling with Adaptive Frequency Steering (IAFS)<n>IAFS addresses the challenge of balancing perceptual quality and structural fidelity by progressively refining the generated image through iterative correction of structural deviations.<n>Experiments show that IAFS effectively resolves the perception-fidelity conflict, yielding consistently improved perceptual detail and structural accuracy, and outperforming existing inference-time scaling methods.
  arXiv  Detail & Related papers  (2025-12-29T15:09:20Z)
• DynaPURLS: Dynamic Refinement of Part-aware Representations for Skeleton-based Zero-Shot Action Recognition [51.80782323686666]
  We introduce textbfDynaPURLS, a unified framework that establishes robust, multi-scale visual-semantic correspondences.<n>Our framework leverages a large language model to generate hierarchical textual descriptions that encompass both global movements and local body-part dynamics.<n>Experiments on three large-scale benchmark datasets, including NTU RGB+D 60/120 and PKU-MMD, demonstrate that DynaPURLS significantly outperforms prior art.
  arXiv  Detail & Related papers  (2025-12-12T10:39:10Z)
• Edit2Perceive: Image Editing Diffusion Models Are Strong Dense Perceivers [55.15722080205737]
  Edit2Perceive is a unified diffusion framework that adapts editing models for depth, normal, and matting.<n>Our single-step deterministic inference yields up to faster runtime while training on relatively small datasets.
  arXiv  Detail & Related papers  (2025-11-24T01:13:51Z)
• WavefrontDiffusion: Dynamic Decoding Schedule or Improved Reasoning [39.57157800995735]
  We present WavefrontDiffusion, a dynamic decoding approach that expands a wavefront of active tokens outward from finalized positions.<n>Across four benchmarks in reasoning and code generation, WavefrontDiffusion achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-11-22T07:33:00Z)
• Be Decisive: Noise-Induced Layouts for Multi-Subject Generation [56.80513553424086]
  Complex prompts lead to subject leakage, causing inaccuracies in quantities, attributes, and visual features.<n>We introduce a new approach that predicts a spatial layout aligned with the prompt, derived from the initial noise, and refines it throughout the denoising process.<n>Our method employs a small neural network to predict and refine the evolving noise-induced layout at each denoising step.
  arXiv  Detail & Related papers  (2025-05-27T17:54:24Z)
• Latent Convergence Modulation in Large Language Models: A Novel Approach to Iterative Contextual Realignment [0.0]
  A structured modulation mechanism was introduced to regulate hidden state transitions.<n>Lattice adjustments contributed to reductions in perplexity fluctuations, entropy variance, and lexical instability.
  arXiv  Detail & Related papers  (2025-02-10T09:46:33Z)
• Context-Preserving Gradient Modulation for Large Language Models: A Novel Approach to Semantic Consistency in Long-Form Text Generation [0.19791587637442667]
  A novel modulation gradient approach is introduced to adjust parameter updates dynamically in response to contextual relevance.<n>The proposed method enhances the stability of model-generated narratives without imposing significant computational overhead.
  arXiv  Detail & Related papers  (2025-02-05T22:13:06Z)
• Noise Diffusion for Enhancing Semantic Faithfulness in Text-to-Image Synthesis [19.109259539368]
  This paper proposes leveraging the language comprehension capabilities of large vision-language models to guide the optimization of the initial noisy latent.<n>We introduce the Noise Diffusion process, which updates the noisy latent to generate semantically faithful images while preserving distribution consistency.<n> Experimental results demonstrate the effectiveness and adaptability of our framework, consistently enhancing semantic alignment across various diffusion models.
  arXiv  Detail & Related papers  (2024-11-25T15:40:47Z)
• Structure Matters: Tackling the Semantic Discrepancy in Diffusion Models for Image Inpainting [18.773724519657446]
  Denoising diffusion probabilistic models for image inpainting aim to add the noise to the texture of image during the forward process and recover masked regions with unmasked ones of the texture via the reverse denoising process. Despite the meaningful semantics generation, the existing arts suffer from the semantic discrepancy between masked and unmasked regions, since the semantically dense unmasked texture fails to be completely degraded while the masked regions turn to the pure noise in diffusion process,leading to the large discrepancy between them. We propose a novel structure-guided diffusion model named StrDiffusion,to reformulate the conventional texture denoising process under
  arXiv  Detail & Related papers  (2024-03-29T00:40:12Z)
• High-Fidelity Speech Synthesis with Minimal Supervision: All Using Diffusion Models [56.00939852727501]
  Minimally-supervised speech synthesis decouples TTS by combining two types of discrete speech representations. Non-autoregressive framework enhances controllability, and duration diffusion model enables diversified prosodic expression.
  arXiv  Detail & Related papers  (2023-09-27T09:27:03Z)
• SeqDiffuSeq: Text Diffusion with Encoder-Decoder Transformers [50.90457644954857]
  In this work, we apply diffusion models to approach sequence-to-sequence text generation. We propose SeqDiffuSeq, a text diffusion model for sequence-to-sequence generation. Experiment results illustrate the good performance on sequence-to-sequence generation in terms of text quality and inference time.
  arXiv  Detail & Related papers  (2022-12-20T15:16:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.