Towards Source-Aware Object Swapping with Initial Noise Perturbation

• URL: http://arxiv.org/abs/2602.23697v2
• Date: Mon, 02 Mar 2026 14:47:58 GMT
• Title: Towards Source-Aware Object Swapping with Initial Noise Perturbation
• Authors: Jiahui Zhan, Xianbing Sun, Xiangnan Zhu, Yikun Ji, Ruitong Liu, Liqing Zhang, Jianfu Zhang,
• Abstract summary: SourceSwap is a self-supervised and source-aware framework that learns cross-object alignment.<n>We train a dual U-Net with full-source conditioning and a noise-free reference encoder, enabling direct inter-object alignment.<n> Experiments demonstrate that SourceSwap achieves superior fidelity, stronger scene preservation, and more natural harmony.
• Score: 10.974803680416876
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Object swapping aims to replace a source object in a scene with a reference object while preserving object fidelity, scene fidelity, and object-scene harmony. Existing methods either require per-object finetuning and slow inference or rely on extra paired data that mostly depict the same object across contexts, forcing models to rely on background cues rather than learning cross-object alignment. We propose SourceSwap, a self-supervised and source-aware framework that learns cross-object alignment. Our key insight is to synthesize high-quality pseudo pairs from any image via a frequency-separated perturbation in the initial-noise space, which alters appearance while preserving pose, coarse shape, and scene layout, requiring no videos, multi-view data, or additional images. We then train a dual U-Net with full-source conditioning and a noise-free reference encoder, enabling direct inter-object alignment, zero-shot inference without per-object finetuning, and lightweight iterative refinement. We further introduce SourceBench, a high-quality benchmark with higher resolution, more categories, and richer interactions. Experiments demonstrate that SourceSwap achieves superior fidelity, stronger scene preservation, and more natural harmony, and it transfers well to edits such as subject-driven refinement and face swapping.

Related papers

• PLACID: Identity-Preserving Multi-Object Compositing via Video Diffusion with Synthetic Trajectories [22.63777279327245]
  PLACID is a framework that transforms a collection of object images into an appealing multi-object composite.<n>First, we leverage a pretrained image-to-video (I2V) diffusion model with text control to preserve objects consistency, identities, and background details.<n>Second, we propose a novel data curation strategy that generates synthetic sequences where randomly placed objects smoothly move to their target positions.
  arXiv  Detail & Related papers  (2026-01-30T19:42:54Z)
• Learning Object-Centric Representations Based on Slots in Real World Scenarios [5.922488908114023]
  This thesis introduces a framework that adapts powerful pretrained diffusion models for object-centric synthesis.<n>We identify a core challenge: balancing global scene coherence with disentangled object control.<n>Our method integrates lightweight, slot-based conditioning into pretrained models, preserving their visual priors while providing object-specific manipulation.
  arXiv  Detail & Related papers  (2025-09-29T12:01:49Z)
• ObjectGS: Object-aware Scene Reconstruction and Scene Understanding via Gaussian Splatting [54.92763171355442]
  ObjectGS is an object-aware framework that unifies 3D scene reconstruction with semantic understanding.<n>We show through experiments that ObjectGS outperforms state-of-the-art methods on open-vocabulary and panoptic segmentation tasks.
  arXiv  Detail & Related papers  (2025-07-21T10:06:23Z)
• ObjectMover: Generative Object Movement with Video Prior [69.75281888309017]
  We present ObjectMover, a generative model that can perform object movement in challenging scenes.<n>We show that with this approach, our model is able to adjust to complex real-world scenarios.<n>We propose a multi-task learning strategy that enables training on real-world video data to improve the model generalization.
  arXiv  Detail & Related papers  (2025-03-11T04:42:59Z)
• DiffUHaul: A Training-Free Method for Object Dragging in Images [78.93531472479202]
  We propose a training-free method, dubbed DiffUHaul, for the object dragging task. We first apply attention masking in each denoising step to make the generation more disentangled across different objects. In the early denoising steps, we interpolate the attention features between source and target images to smoothly fuse new layouts with the original appearance.
  arXiv  Detail & Related papers  (2024-06-03T17:59:53Z)
• ROAM: Robust and Object-Aware Motion Generation Using Neural Pose Descriptors [73.26004792375556]
  This paper shows that robustness and generalisation to novel scene objects in 3D object-aware character synthesis can be achieved by training a motion model with as few as one reference object. We leverage an implicit feature representation trained on object-only datasets, which encodes an SE(3)-equivariant descriptor field around the object. We demonstrate substantial improvements in 3D virtual character motion and interaction quality and robustness to scenarios with unseen objects.
  arXiv  Detail & Related papers  (2023-08-24T17:59:51Z)
• AnyDoor: Zero-shot Object-level Image Customization [63.44307304097742]
  This work presents AnyDoor, a diffusion-based image generator with the power to teleport target objects to new scenes at user-specified locations. Our model is trained only once and effortlessly generalizes to diverse object-scene combinations at the inference stage.
  arXiv  Detail & Related papers  (2023-07-18T17:59:02Z)
• ObjectStitch: Generative Object Compositing [43.206123360578665]
  We propose a self-supervised framework for object compositing using conditional diffusion models. Our framework can transform the viewpoint, geometry, color and shadow of the generated object while requiring no manual labeling. Our method outperforms relevant baselines in both realism and faithfulness of the synthesized result images in a user study on various real-world images.
  arXiv  Detail & Related papers  (2022-12-02T02:15:13Z)
• Object-Centric Image Generation from Layouts [93.10217725729468]
  We develop a layout-to-image-generation method to generate complex scenes with multiple objects. Our method learns representations of the spatial relationships between objects in the scene, which lead to our model's improved layout-fidelity. We introduce SceneFID, an object-centric adaptation of the popular Fr'echet Inception Distance metric, that is better suited for multi-object images.
  arXiv  Detail & Related papers  (2020-03-16T21:40:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.