PICS: Pairwise Image Compositing with Spatial Interactions

• URL: http://arxiv.org/abs/2603.06873v1
• Date: Fri, 06 Mar 2026 20:48:21 GMT
• Title: PICS: Pairwise Image Compositing with Spatial Interactions
• Abstract summary: PICS is a self-supervised composition-by-decomposition paradigm that composes objects in parallel.<n>An Interaction Transformer employs mask-guided Mixture-of-Experts to route background, exclusive, and overlap regions.<n>To further enhance robustness to geometric variations, we incorporate geometry-aware augmentations covering both out-of-plane and in-plane pose changes.
• Score: 20.306534705755613
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite strong single-turn performance, diffusion-based image compositing often struggles to preserve coherent spatial relations in pairwise or sequential edits, where subsequent insertions may overwrite previously generated content and disrupt physical consistency. We introduce PICS, a self-supervised composition-by-decomposition paradigm that composes objects in parallel while explicitly modeling the compositional interactions among (fully-/partially-)visible objects and background. At its core, an Interaction Transformer employs mask-guided Mixture-of-Experts to route background, exclusive, and overlap regions to dedicated experts, with an adaptive α-blending strategy that infers a compatibility-aware fusion of overlapping objects while preserving boundary fidelity. To further enhance robustness to geometric variations, we incorporate geometry-aware augmentations covering both out-of-plane and in-plane pose changes of objects. Our method delivers superior pairwise compositing quality and substantially improved stability, with extensive evaluations across virtual try-on, indoor, and street scene settings showing consistent gains over state-of-the-art baselines. Code and data are available at https://github.com/RyanHangZhou/PICS

Related papers

• To Blend In, First Decouple: Rethinking Camouflage Image Generation via Context-Decoupled Representations [18.9512030963437]
  Camouflage image generation (CIG) focuses on generating visually concealed objects that seamlessly blend into their backgrounds.<n>Existing methods typically follow either background-guided paradigms that adapt object appearance via style transfer, or foreground-guided strategies that outpaint surrounding regions conditioned on object features.<n>We propose a new context-decoupled generative paradigm, termed CamoDreamer, which aims to isolate contextual conditional guidance and explicitly decouple latent camouflage features into coordinated object and background control streams.
  arXiv  Detail & Related papers  (2026-07-20T10:01:36Z)
• Beyond Point-Wise Matching: Structural Representation Alignment for Accelerating Diffusion Transformers [93.3976834364707]
  We propose sREPA, a structural REPresentation Alignment framework to enforce consistency in the relational geometry of feature maps.<n>By encouraging the model to internalize holistic spatial layouts and structural correlations from pre-trained features, sREPA achieves faster and more stable convergence.
  arXiv  Detail & Related papers  (2026-05-16T12:01:04Z)
• Interact3D: Compositional 3D Generation of Interactive Objects [31.12099147294145]
  We present a novel framework designed to generate physically plausible interacting 3D compositional objects.<n>Our approach first leverages advanced generative priors to curate high-quality individual assets.<n>To physically compose these assets, we then introduce a robust two-stage composition pipeline.
  arXiv  Detail & Related papers  (2026-03-17T03:21:06Z)
• ControlVP: Interactive Geometric Refinement of AI-Generated Images with Consistent Vanishing Points [32.23473666846317]
  We propose ControlVP, a user-guided framework for correcting vanishing point inconsistencies in generated images.<n>Our approach extends a pre-trained diffusion model by incorporating structural guidance derived from building contours.<n>Our method enhances global geometric consistency while maintaining visual fidelity comparable to the baselines.
  arXiv  Detail & Related papers  (2025-12-08T12:38:11Z)
• Follow My Hold: Hand-Object Interaction Reconstruction through Geometric Guidance [61.41904916189093]
  We propose a novel diffusion-based framework for reconstructing 3D geometry of hand-held objects from monocular RGB images.<n>We use hand-object interaction as geometric guidance to ensure plausible hand-object interactions.
  arXiv  Detail & Related papers  (2025-08-25T17:11:53Z)
• Aligned Novel View Image and Geometry Synthesis via Cross-modal Attention Instillation [62.87088388345378]
  We introduce a diffusion-based framework that performs aligned novel view image and geometry generation via a warping-and-inpainting methodology.<n>Method leverages off-the-shelf geometry predictors to predict partial geometries viewed from reference images.<n>Cross-modal attention distillation is proposed to ensure accurate alignment between generated images and geometry.
  arXiv  Detail & Related papers  (2025-06-13T16:19:00Z)
• Geometry-Editable and Appearance-Preserving Object Compositon [67.98806888489385]
  General object composition (GOC) aims to seamlessly integrate a target object into a background scene with desired geometric properties.<n>Recent approaches derive semantic embeddings and integrate them into advanced diffusion models to enable geometry-editable generation.<n>We introduce a Disentangled Geometry-editable and Appearance-preserving Diffusion model that first leverages semantic embeddings to implicitly capture desired geometric transformations.
  arXiv  Detail & Related papers  (2025-05-27T09:05:28Z)
• ShapeShift: Towards Text-to-Shape Arrangement Synthesis with Content-Aware Geometric Constraints [13.2441524021269]
  ShapeShift is a text-guided image-to-image translation task that requires rearranging the input set of rigid shapes into non-overlapping configurations.<n>We introduce a content-aware collision resolution mechanism that applies minimal semantically coherent adjustments when overlaps occur.<n>Our approach yields interpretable compositions where spatial relationships clearly embody the textual prompt.
  arXiv  Detail & Related papers  (2025-03-18T20:48:58Z)
• Hierarchical Context Alignment with Disentangled Geometric and Temporal Modeling for Semantic Occupancy Prediction [61.484280369655536]
  Camera-based 3D Semantic Occupancy Prediction (SOP) is crucial for understanding complex 3D scenes from limited 2D image observations.<n>Existing SOP methods typically aggregate contextual features to assist the occupancy representation learning.<n>We introduce a new Hierarchical context alignment paradigm for a more accurate SOP (Hi-SOP)
  arXiv  Detail & Related papers  (2024-12-11T09:53:10Z)
• Enhancing Object Coherence in Layout-to-Image Synthesis [2.3805413240744304]
  We propose a novel diffusion model with effective global semantic fusion (GSF) and self-similarity feature enhancement modules to guide the object coherence.<n>For semantic coherence, we argue that the image caption contains rich information for defining the semantic relationship within the objects in the images.<n>To improve the physical coherence, we develop a Self-similarity Coherence Attention synthesis (SCA) module to explicitly integrate local contextual physical coherence relation into each pixel's generation process.
  arXiv  Detail & Related papers  (2023-11-17T13:43:43Z)
• Image Morphing with Perceptual Constraints and STN Alignment [70.38273150435928]
  We propose a conditional GAN morphing framework operating on a pair of input images. A special training protocol produces sequences of frames, combined with a perceptual similarity loss, promote smooth transformation over time. We provide comparisons to classic as well as latent space morphing techniques, and demonstrate that, given a set of images for self-supervision, our network learns to generate visually pleasing morphing effects.
  arXiv  Detail & Related papers  (2020-04-29T10:49:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.