Training-Free Inpainting Across Domains with a Frozen Text-to-Image Diffusion Model

• URL: http://arxiv.org/abs/2609.00862v1
• Date: Tue, 01 Sep 2026 07:57:53 GMT
• Title: Training-Free Inpainting Across Domains with a Frozen Text-to-Image Diffusion Model
• Abstract summary: We show that a frozen generic text-to-image diffusion model can perform conditional inpainting across three evaluated natural-image domains.<n>Step-PI augments known-region projection with boundary-interior latent feedback, persistent PI state, and a four-field release schedule.
• Score: 0.8716913598251385
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We show that a frozen generic text-to-image diffusion model can perform conditional inpainting across three evaluated natural-image domains with one fixed controller configuration, without inpainting-specific weight training, dataset-specific weight adaptation, or learned inpainting-specific conditioning channels. Step-PI augments known-region projection with boundary-interior latent feedback, persistent PI state, and a predefined four-field release schedule that modulates controller signals along the reverse trajectory. Developed only on Main35-disjoint CelebA-HQ pilots, the controller transfers unchanged to AFHQ and Places2. Across two field-identical comparisons on the same 3,500 cases, adding persistent state and replacing uniform release with the predefined schedule each improve all 15 dataset-metric cells; 95% bootstrap intervals exclude zero for all five metrics in both comparisons. In descriptive native-route comparisons, Step-PI leads LanPaint and PILOT (the closest evaluated training-free baselines using vanilla SD1.5) on all five equal-dataset macro metrics. Inpainting-trained systems retain the absolute metric leads but rely on substantial inpainting-specific offline optimization. Our method provides a complementary approach for repurposing a frozen generic text-to-image model for cross-domain inpainting through test-time latent control.

Related papers

• Logit Refiner: Improving Visual Autoregressive Models via Intra-Scale Dependency Modeling [21.906953897227442]
  Visual Autoregressive Models generate images through next-scale prediction.<n>Mean-field approximation discards spatial dependencies among same-scale tokens.<n>Logit Refiner restores intra-scale dependencies by sequentially sampling tokens conditioned on frozen backbone features.
  arXiv  Detail & Related papers  (2026-09-10T16:57:05Z)
• Multimodal-Multiresolution Foundation Model for Lunar Remote Sensing [5.471205088925021]
  We present a multimodal foundation model for lunar remote sensing, pretrained from scratch on SomBench.<n>The model adapts the TerraMind masked-token architecture with two lunar-specific extensions.<n>We evaluate on four benchmarks: crater detection at WAC and NAC scales, irregular mare patch (IMP) segmentation, and polar ice prospectivity regression.
  arXiv  Detail & Related papers  (2026-09-08T14:55:35Z)
• Structured-Prior-Guided Diffusion Inpainting with Physical Consistency for Traffic Sign Augmentation [12.366721313845483]
  General-purpose inpainting models distort digits, deform geometry and perspective, and shift colours when applied directly to sign regions.<n>We propose a structured-prior-guided diffusion inpainting framework with physical consistency.<n>We train by self-supervised reconstruction on a large set of images collected in-house at AMAP, then evaluate zero-shot on the public TT100K-2021 dataset.
  arXiv  Detail & Related papers  (2026-09-02T09:22:23Z)
• Generative Semantic Segmentation via an Observable Semantic-Image Interface and Hierarchical Generator Evidence Alignment [1.6199400106794555]
  Generative semantic segmentation exposes structured predictions as images, but direct color decoding is susceptible to color drift and boundary mixing.<n>We present Semantic Prism, a conditional semantic-image generation-and-refinement framework with inference.
  arXiv  Detail & Related papers  (2026-08-12T01:00:04Z)
• FoundationGeo: Learning Spatial Pixel-Wise Fields for Monocular Metric Geometry [52.232826445242644]
  FoundationGeo bridges relative and metric prediction via spatial calibration and principled data design.<n>Stage 1 learns a high-fidelity, affine-invariant geometry model by initializing with DINOv3.<n>Stage 2 moves beyond global scaling by introducing pixel-wise calibration fields for metric estimation.
  arXiv  Detail & Related papers  (2026-07-13T14:10:01Z)
• Tensor-Based Batch Fuzzing with Adaptive Perturbation Scaling for Deep Neural Networks [7.190532917825181]
  We present a new specification-aware batch fuzzing framework with adaptive perturbation scaling.<n>Our framework achieves up to 40X higher throughput and 4X more violations than the sequential baseline under the same time budget.<n>We evaluate our framework extensively on three benchmarks, covering six networks and over 400 specifications across TrafficSigns, Cifar100, and TinyImageNet.
  arXiv  Detail & Related papers  (2026-06-23T23:56:21Z)
• Surflo: Consistent 3D Surface Flow Model with Global State [57.57234680235885]
  Surflo decodes oriented 3D surface points by independently transporting them from noise onto the surface via flow matching.<n>To suppress the local inconsistencies inherent to independent per-point decoding, an inference-time guidance term correlates nearby points by injecting a photometric gradient.<n>Surflo matches or surpasses feed-forward baselines on surface metrics, runs an order of magnitude faster than optimization-based methods that require hundreds of views, and is the only feed-forward approach to combine a global latent with arbitrary-resolution decoding.
  arXiv  Detail & Related papers  (2026-06-11T17:48:38Z)
• SIGMA: Semantic-Difference Instruction-Grounding Mask Annotator for Text-Driven Image Manipulation Localization [64.6372217552272]
  Public editing datasets contain millions of structurally identical (original, edited) pairs to IML training samples, lacking only pixel-level masks.<n>We propose SIGMA, which performs semantic-feature differencing in a vision foundation backbone.<n>It produces a 1.1M IML training set that improves six diverse detectors by +18.34% F1 across five datasets.
  arXiv  Detail & Related papers  (2026-05-27T03:55:13Z)
• OmniLiDAR: A Unified Diffusion Framework for Multi-Domain 3D LiDAR Generation [100.36917520403227]
  We present a unified text-conditioned diffusion framework that generates LiDAR scans in a shared range-image representation across eight representative domains.<n>In the absence of a public consolidated benchmark, we construct an 8-domain dataset by combining real-world scans with physically based weather simulation and systematic beam reduction.
  arXiv  Detail & Related papers  (2026-05-13T17:42:20Z)
• Mitigating Long-Tail Bias via Prompt-Controlled Diffusion Augmentation [36.94429692322632]
  We present a prompt-controlled diffusion augmentation framework that synthesizes paired label--image samples with explicit control of both domain and semantic composition.<n>We show gains concentrated on minority classes and improved Urban and Rural generalization, demonstrating controllable augmentation as a practical mechanism to mitigate long-tail bias in remote-sensing segmentation.
  arXiv  Detail & Related papers  (2026-02-04T16:49:16Z)
• XD-MAP: Cross-Modal Domain Adaptation using Semantic Parametric Mapping [5.609281438287908]
  We propose a novel approach to transferring sensor-specific knowledge from an image dataset to LiDAR.<n>Our method XD-MAP leverages detections from a neural network on camera images to create a semantic parametric map.<n>The results demonstrate the effectiveness of our approach achieving strong performance on LiDAR data without any manual labeling.
  arXiv  Detail & Related papers  (2026-01-20T21:00:26Z)
• Dynamic Classifier-Free Diffusion Guidance via Online Feedback [53.54876309092376]
  "One-size-all" approach fails to adapt to the diverse requirements of different prompts.<n>We introduce a framework for dynamic CFG scheduling.<n>We demonstrate the effectiveness of our approach on both small-scale models and the state-of-the-art Imagen 3.
  arXiv  Detail & Related papers  (2025-09-19T16:27:19Z)
• Leveraging Semantic Attribute Binding for Free-Lunch Color Control in Diffusion Models [53.73253164099701]
  We introduce ColorWave, a training-free approach that achieves exact RGB-level color control in diffusion models without fine-tuning.<n>We demonstrate that ColorWave establishes a new paradigm for structured, color-consistent diffusion-based image synthesis.
  arXiv  Detail & Related papers  (2025-03-12T21:49:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.