HomeDiffusion: Zero-Shot Object Customization with Multi-View Representation Learning for Indoor Scenes

• URL: http://arxiv.org/abs/2606.29828v1
• Date: Mon, 29 Jun 2026 06:16:09 GMT
• Title: HomeDiffusion: Zero-Shot Object Customization with Multi-View Representation Learning for Indoor Scenes
• Abstract summary: We introduce HomeDiffusion, which can leverage multi-viewpoint images of the same reference object to generate visually harmonious object poses.<n>Based on diffusion models, we introduce HomeDiffusion, which can leverage multi-viewpoint images of the same reference object to accurately generate visually harmonious object poses.
• Score: 6.582546260109086
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recently, zero-shot object customization generation methods have rapidly developed and shown tremendous potential for applications. For instance, in the e-commerce domain, consumers can observe the visual effect of furniture placed within their personal living spaces or clothes worn on their own bodies. Many existing approaches perform object customization generation based on diffusion models and extracted reference object features. However, the generated object significantly diverges from the original reference object in details such as patterns and curves. Particularly for asymmetrical reference objects, the absence of comprehensive multi-viewpoint information prevents the generation of object poses that harmonize with the background scene. To address these shortcomings, we have constructed a novel dataset comprising multi-angle images of furniture and indoor scenes. Based on diffusion models, we introduce HomeDiffusion, which can leverage multi-viewpoint images of the same reference object to accurately generate visually harmonious object poses within specified areas of the background scene. During the diffusion process, we further extract high-fidelity details of the reference object and perform cross-attention with the noise latents in the latent space, thereby ensuring the preservation of details in the customized object generation. Extensive qualitative and quantitative experiments demonstrate that our method achieves superior performance over other existing zero-shot as well as few-shot object customization approaches.

Related papers

• Towards Source-Aware Object Swapping with Initial Noise Perturbation [10.974803680416876]
  SourceSwap is a self-supervised and source-aware framework that learns cross-object alignment.<n>We train a dual U-Net with full-source conditioning and a noise-free reference encoder, enabling direct inter-object alignment.<n> Experiments demonstrate that SourceSwap achieves superior fidelity, stronger scene preservation, and more natural harmony.
  arXiv  Detail & Related papers  (2026-02-27T05:54:29Z)
• FROMAT: Multiview Material Appearance Transfer via Few-Shot Self-Attention Adaptation [49.74776147964999]
  We present a lightweight adaptation technique for appearance transfer in multiview diffusion models.<n>Our method learns to combine object identity from an input image with appearance cues rendered in a separate reference image, producing multi-view-consistent output.
  arXiv  Detail & Related papers  (2025-12-10T13:06:40Z)
• ObjectClear: Complete Object Removal via Object-Effect Attention [56.2893552300215]
  We introduce a new dataset for OBject-Effect Removal, named OBER, which provides paired images with and without object effects, along with precise masks for both objects and their associated visual artifacts.<n>We propose a novel framework, ObjectClear, which incorporates an object-effect attention mechanism to guide the model toward the foreground removal regions by learning attention masks.<n>Experiments demonstrate that ObjectClear outperforms existing methods, achieving improved object-effect removal quality and background fidelity, especially in complex scenarios.
  arXiv  Detail & Related papers  (2025-05-28T17:51:17Z)
• Salient Object-Aware Background Generation using Text-Guided Diffusion Models [4.747826159446815]
  We present a model for adapting inpainting diffusion models to the salient object outpainting task using Stable Diffusion and ControlNet architectures. Our proposed approach reduces object expansion by 3.6x on average with no degradation in standard visual metrics across multiple datasets.
  arXiv  Detail & Related papers  (2024-04-15T22:13:35Z)
• Object-Driven One-Shot Fine-tuning of Text-to-Image Diffusion with Prototypical Embedding [7.893308498886083]
  Our proposed method aims to address the challenges of generalizability and fidelity in an object-driven way. A prototypical embedding is based on the object's appearance and its class, before fine-tuning the diffusion model. Our method outperforms several existing works.
  arXiv  Detail & Related papers  (2024-01-28T17:11:42Z)
• AnyDoor: Zero-shot Object-level Image Customization [63.44307304097742]
  This work presents AnyDoor, a diffusion-based image generator with the power to teleport target objects to new scenes at user-specified locations. Our model is trained only once and effortlessly generalizes to diverse object-scene combinations at the inference stage.
  arXiv  Detail & Related papers  (2023-07-18T17:59:02Z)
• Spotlight Attention: Robust Object-Centric Learning With a Spatial Locality Prior [88.9319150230121]
  Object-centric vision aims to construct an explicit representation of the objects in a scene. We incorporate a spatial-locality prior into state-of-the-art object-centric vision models. We obtain significant improvements in segmenting objects in both synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2023-05-31T04:35:50Z)
• Salient Objects in Clutter [130.63976772770368]
  This paper identifies and addresses a serious design bias of existing salient object detection (SOD) datasets. This design bias has led to a saturation in performance for state-of-the-art SOD models when evaluated on existing datasets. We propose a new high-quality dataset and update the previous saliency benchmark.
  arXiv  Detail & Related papers  (2021-05-07T03:49:26Z)
• Object-Centric Image Generation from Layouts [93.10217725729468]
  We develop a layout-to-image-generation method to generate complex scenes with multiple objects. Our method learns representations of the spatial relationships between objects in the scene, which lead to our model's improved layout-fidelity. We introduce SceneFID, an object-centric adaptation of the popular Fr'echet Inception Distance metric, that is better suited for multi-object images.
  arXiv  Detail & Related papers  (2020-03-16T21:40:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.