The Unreasonable Effectiveness of Text Embedding Interpolation for Continuous Image Steering

• URL: http://arxiv.org/abs/2603.17998v1
• Date: Wed, 18 Mar 2026 17:57:53 GMT
• Title: The Unreasonable Effectiveness of Text Embedding Interpolation for Continuous Image Steering
• Abstract summary: We present a training-free framework for continuous and controllable image editing at test time for text-conditioned generative models.<n>We find that a simple steering in the text-embedding space is sufficient to produce smooth edit control.<n>Our approach is comparable to training-based alternatives, outperforming other training-free methods.
• Score: 18.29130390175963
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a training-free framework for continuous and controllable image editing at test time for text-conditioned generative models. In contrast to prior approaches that rely on additional training or manual user intervention, we find that a simple steering in the text-embedding space is sufficient to produce smooth edit control. Given a target concept (e.g., enhancing photorealism or changing facial expression), we use a large language model to automatically construct a small set of debiased contrastive prompt pairs, from which we compute a steering vector in the generator's text-encoder space. We then add this vector directly to the input prompt representation to control generation along the desired semantic axis. To obtain a continuous control, we propose an elastic range search procedure that automatically identifies an effective interval of steering magnitudes, avoiding both under-steering (no-edit) and over-steering (changing other attributes). Adding the scaled versions of the same vector within this interval yields smooth and continuous edits. Since our method modifies only textual representations, it naturally generalizes across text-conditioned modalities, including image and video generation. To quantify the steering continuity, we introduce a new evaluation metric that measures the uniformity of semantic change across edit strengths. We compare the continuous editing behavior across methods and find that, despite its simplicity and lightweight design, our approach is comparable to training-based alternatives, outperforming other training-free methods.

Related papers

• Continuous Control of Editing Models via Adaptive-Origin Guidance [27.623081798640907]
  Diffusion-based editing models have emerged as a powerful tool for semantic and image editing.<n>We introduce Adaptive-Origin Guidance (AdaOr), a method that adjusts the standard guidance origin with an identity-conditioned adaptive origin.<n>We evaluate our method on image and video editing tasks, demonstrating that it provides smoother and more consistent control compared to current slider-based editing approaches.
  arXiv  Detail & Related papers  (2026-02-03T18:33:39Z)
• Localized Control in Diffusion Models via Latent Vector Prediction [2.4923006485141284]
  We propose a novel methodology to enable precise local control over user-defined regions of an image.<n>Our method effectively synthesizes high-quality images with controlled local conditions.
  arXiv  Detail & Related papers  (2026-02-02T11:47:48Z)
• Kontinuous Kontext: Continuous Strength Control for Instruction-based Image Editing [76.44219733285898]
  Kontinuous Kontext is an instruction-driven editing model that provides a new dimension of control over edit strength.<n>A lightweight projector network maps the input scalar and the edit instruction to coefficients in the model's modulation space.<n>For training our model, we synthesize a diverse dataset of image-edit-instruction-strength quadruplets using existing generative models.
  arXiv  Detail & Related papers  (2025-10-09T17:51:03Z)
• SAEdit: Token-level control for continuous image editing via Sparse AutoEncoder [52.754326452329956]
  We introduce a method for disentangled and continuous editing through token-level manipulation of text embeddings.<n>The edits are applied by manipulating the embeddings along carefully chosen directions, which control the strength of the target attribute.<n>Our method operates directly on text embeddings without modifying the diffusion process, making it model agnostic and broadly applicable to various image backbones.
  arXiv  Detail & Related papers  (2025-10-06T17:51:04Z)
• Towards Transformer-Based Aligned Generation with Self-Coherence Guidance [51.42269790543461]
  We introduce a training-free approach for enhancing alignment in Transformer-based Text-Guided Diffusion Models (TGDMs)<n>Existing TGDMs often struggle to generate semantically aligned images, particularly when dealing with complex text prompts or multi-concept attribute binding challenges.<n>Our method addresses these challenges by directly optimizing cross-attention maps during the generation process.
  arXiv  Detail & Related papers  (2025-03-22T07:03:57Z)
• Textual and Visual Prompt Fusion for Image Editing via Step-Wise Alignment [10.82748329166797]
  We propose a framework that integrates a fusion of generated visual references and text guidance into the semantic latent space of a textitfrozen pre-trained diffusion model.<n>Using only a tiny neural network, our framework provides control over diverse content and attributes, driven intuitively by the text prompt.
  arXiv  Detail & Related papers  (2023-08-30T08:40:15Z)
• LRANet: Towards Accurate and Efficient Scene Text Detection with Low-Rank Approximation Network [63.554061288184165]
  We propose a novel parameterized text shape method based on low-rank approximation. By exploring the shape correlation among different text contours, our method achieves consistency, compactness, simplicity, and robustness in shape representation. We implement an accurate and efficient arbitrary-shaped text detector named LRANet.
  arXiv  Detail & Related papers  (2023-06-27T02:03:46Z)
• Being Comes from Not-being: Open-vocabulary Text-to-Motion Generation with Wordless Training [178.09150600453205]
  In this paper, we investigate offline open-vocabulary text-to-motion generation in a zero-shot learning manner. Inspired by the prompt learning in NLP, we pretrain a motion generator that learns to reconstruct the full motion from the masked motion. Our method reformulates the input text into a masked motion as the prompt for the motion generator to reconstruct'' the motion.
  arXiv  Detail & Related papers  (2022-10-28T06:20:55Z)
• Text Revision by On-the-Fly Representation Optimization [76.11035270753757]
  Current state-of-the-art methods formulate these tasks as sequence-to-sequence learning problems. We present an iterative in-place editing approach for text revision, which requires no parallel data. It achieves competitive and even better performance than state-of-the-art supervised methods on text simplification.
  arXiv  Detail & Related papers  (2022-04-15T07:38:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.