Geometry-Editable and Appearance-Preserving Object Compositon

• URL: http://arxiv.org/abs/2505.20914v1
• Date: Tue, 27 May 2025 09:05:28 GMT
• Title: Geometry-Editable and Appearance-Preserving Object Compositon
• Abstract summary: General object composition (GOC) aims to seamlessly integrate a target object into a background scene with desired geometric properties.<n>Recent approaches derive semantic embeddings and integrate them into advanced diffusion models to enable geometry-editable generation.<n>We introduce a Disentangled Geometry-editable and Appearance-preserving Diffusion model that first leverages semantic embeddings to implicitly capture desired geometric transformations.
• Score: 67.98806888489385
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: General object composition (GOC) aims to seamlessly integrate a target object into a background scene with desired geometric properties, while simultaneously preserving its fine-grained appearance details. Recent approaches derive semantic embeddings and integrate them into advanced diffusion models to enable geometry-editable generation. However, these highly compact embeddings encode only high-level semantic cues and inevitably discard fine-grained appearance details. We introduce a Disentangled Geometry-editable and Appearance-preserving Diffusion (DGAD) model that first leverages semantic embeddings to implicitly capture the desired geometric transformations and then employs a cross-attention retrieval mechanism to align fine-grained appearance features with the geometry-edited representation, facilitating both precise geometry editing and faithful appearance preservation in object composition. Specifically, DGAD builds on CLIP/DINO-derived and reference networks to extract semantic embeddings and appearance-preserving representations, which are then seamlessly integrated into the encoding and decoding pipelines in a disentangled manner. We first integrate the semantic embeddings into pre-trained diffusion models that exhibit strong spatial reasoning capabilities to implicitly capture object geometry, thereby facilitating flexible object manipulation and ensuring effective editability. Then, we design a dense cross-attention mechanism that leverages the implicitly learned object geometry to retrieve and spatially align appearance features with their corresponding regions, ensuring faithful appearance consistency. Extensive experiments on public benchmarks demonstrate the effectiveness of the proposed DGAD framework.

Related papers

• DeGenseGS: Geometrically and Semantically Decoupled Surgical Scene Understanding in 4D Gaussian Splatting [17.162560675768642]
  Existing solutions integrate Vision-Language Models into deformable fields via a rigid coupling scheme that tightly binds semantic features to geometric warping.<n>We propose DeGenseGS, Geometric Understandingally and Semantically Decoupled Scene in 4D Splatting, a novel framework that independently models semantic evolution and geometric deformation.<n>Our framework yields enhanced geometric robustness and robust semantic-anatomic alignment, enabling spatially continuous segmentation despite drastic tissue deformation and topological transitions.
  arXiv  Detail & Related papers  (2026-07-06T07:56:17Z)
• G$^2$TAM: Geometry Grounded Track Anything Model [49.05553245076823]
  We propose a unified framework for promptable instance tracking in 3D using only unordered RGB images or videos.<n>G$2$TAM employs spatially aligned geometric representations as implicit memory, ensuring stable instance identity and localization across frames and views.<n>At its core is a cross-modal spatial encoder that integrates visual and textual prompts into a shared geometric space, enabling end-to-end spatial reconstruction and instance-consistent mask prediction.
  arXiv  Detail & Related papers  (2026-07-04T09:32:13Z)
• MAGE: View-guided Point Cloud Completion with Efficient Modality Alignment and Adaptive Geometry Enhancement [19.44604826924269]
  View-based point cloud completion aims to recover a complete 3D shape from a partial point cloud, guided by a single-view image.<n>Existing approaches often suffer from limited performance due to weak modality alignment and limited self-geometry enhancement.<n>We propose a unified geometry-aware framework that integrates efficient modality alignment and adaptive geometry enhancement.
  arXiv  Detail & Related papers  (2026-06-30T05:25:29Z)
• Learning 3D Representations for Spatial Intelligence from Unposed Multi-View Images [81.94999489820974]
  UniSplat is a feed-forward framework for learning 3D representations from unposed multi-view images.<n>We introduce a dual-masking strategy that strengthens geometry induction in the encoder.<n>Second, we develop a coarse-to-fine Gaussian splatting strategy that reduces appearance-semantics inconsistencies.<n>Third, we introduce a pose-conditioned recalibration mechanism that interrelates predicted 3D point and semantic maps into the image plane.
  arXiv  Detail & Related papers  (2026-04-12T10:36:18Z)
• Interact3D: Compositional 3D Generation of Interactive Objects [31.12099147294145]
  We present a novel framework designed to generate physically plausible interacting 3D compositional objects.<n>Our approach first leverages advanced generative priors to curate high-quality individual assets.<n>To physically compose these assets, we then introduce a robust two-stage composition pipeline.
  arXiv  Detail & Related papers  (2026-03-17T03:21:06Z)
• Adaptive Scaling with Geometric and Visual Continuity of completed 3D objects [0.0]
  We introduce a part-aware scaling framework that transforms static completed SDFs into structurally coherent objects.<n>Our method overcomes the inherent rigidity of completed SDFs and is visually more appealing than global and naive selective scaling.
  arXiv  Detail & Related papers  (2026-02-13T13:05:54Z)
• Joint Geometry-Appearance Human Reconstruction in a Unified Latent Space via Bridge Diffusion [57.09673862519791]
  This paper introduces textbfJGA-LBD, a novel framework that unifies the modeling of geometry and appearance into a joint latent representation.<n> Experiments demonstrate that JGA-LBD outperforms current state-of-the-art approaches in terms of both geometry fidelity and appearance quality.
  arXiv  Detail & Related papers  (2026-01-01T12:48:56Z)
• ArtiLatent: Realistic Articulated 3D Object Generation via Structured Latents [31.495577251319315]
  ArtiLatent is a generative framework that synthesizes human-made 3D objects with fine-grained geometry, accurate articulation, and realistic appearance.
  arXiv  Detail & Related papers  (2025-10-24T13:08:15Z)
• GeoComplete: Geometry-Aware Diffusion for Reference-Driven Image Completion [36.02469602451232]
  We propose a novel framework that incorporates explicit 3D structural guidance to enforce geometric consistency in completed regions.<n>Experiments show that GeoComplete achieves a 17.1 PSNR improvement over state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-10-03T15:38:12Z)
• Dense Semantic Matching with VGGT Prior [49.42199006453071]
  We propose an approach that retains VGGT's intrinsic strengths by reusing early feature stages, fine-tuning later ones, and adding a semantic head for bidirectional correspondences.<n>Our approach achieves superior geometry awareness, matching reliability, and manifold preservation, outperforming previous baselines.
  arXiv  Detail & Related papers  (2025-09-25T14:56:11Z)
• Follow My Hold: Hand-Object Interaction Reconstruction through Geometric Guidance [61.41904916189093]
  We propose a novel diffusion-based framework for reconstructing 3D geometry of hand-held objects from monocular RGB images.<n>We use hand-object interaction as geometric guidance to ensure plausible hand-object interactions.
  arXiv  Detail & Related papers  (2025-08-25T17:11:53Z)
• ShapeEmbed: a self-supervised learning framework for 2D contour quantification [45.39160205677261]
  We introduce ShapeEmbed, a self-supervised representation learning framework designed to encode the contour of objects in 2D images.<n>Our approach overcomes the limitations of traditional shape descriptors while improving upon existing state-of-the-art autoencoder-based approaches.<n>We demonstrate that the descriptors learned by our framework outperform their competitors in shape classification tasks on natural and biological images.
  arXiv  Detail & Related papers  (2025-07-01T17:55:57Z)
• Aligned Novel View Image and Geometry Synthesis via Cross-modal Attention Instillation [62.87088388345378]
  We introduce a diffusion-based framework that performs aligned novel view image and geometry generation via a warping-and-inpainting methodology.<n>Method leverages off-the-shelf geometry predictors to predict partial geometries viewed from reference images.<n>Cross-modal attention distillation is proposed to ensure accurate alignment between generated images and geometry.
  arXiv  Detail & Related papers  (2025-06-13T16:19:00Z)
• Self-Supervised Multi-Part Articulated Objects Modeling via Deformable Gaussian Splatting and Progressive Primitive Segmentation [23.18517560629462]
  We introduce DeGSS, a unified framework that encodes articulated objects as deformable 3D Gaussian fields, embedding geometry, appearance, and motion in one compact representation.<n>To evaluate generalization and realism, we enlarge the synthetic PartNet-Mobility benchmark and release RS-Art, a real-to-sim dataset that pairs RGB captures with accurately reverse-engineered 3D models.
  arXiv  Detail & Related papers  (2025-06-11T12:32:16Z)
• ShapeShift: Towards Text-to-Shape Arrangement Synthesis with Content-Aware Geometric Constraints [13.2441524021269]
  ShapeShift is a text-guided image-to-image translation task that requires rearranging the input set of rigid shapes into non-overlapping configurations.<n>We introduce a content-aware collision resolution mechanism that applies minimal semantically coherent adjustments when overlaps occur.<n>Our approach yields interpretable compositions where spatial relationships clearly embody the textual prompt.
  arXiv  Detail & Related papers  (2025-03-18T20:48:58Z)
• JADE: Joint-aware Latent Diffusion for 3D Human Generative Modeling [62.77347895550087]
  We introduce JADE, a generative framework that learns the variations of human shapes with fined-grained control.<n>Our key insight is a joint-aware latent representation that decomposes human bodies into skeleton structures.<n>To generate coherent and plausible human shapes under our proposed decomposition, we also present a cascaded pipeline.
  arXiv  Detail & Related papers  (2024-12-29T14:18:35Z)
• GREAT: Geometry-Intention Collaborative Inference for Open-Vocabulary 3D Object Affordance Grounding [53.42728468191711]
  Open-Vocabulary 3D object affordance grounding aims to anticipate action possibilities'' regions on 3D objects with arbitrary instructions.<n>We propose GREAT (GeometRy-intEntion collAboraTive inference) for Open-Vocabulary 3D Object Affordance Grounding.
  arXiv  Detail & Related papers  (2024-11-29T11:23:15Z)
• Deep Deformable Models: Learning 3D Shape Abstractions with Part Consistency [37.28811220509584]
  Recent methods learn to represent an object shape using a set of simple primitives to fit the target. These primitives do not always correspond to real parts or lack geometric flexibility for semantic interpretation. In this paper, we investigate salient and efficient primitive descriptors for accurate shape abstractions.
  arXiv  Detail & Related papers  (2023-09-02T23:18:28Z)
• Learning to Complete Object Shapes for Object-level Mapping in Dynamic Scenes [30.500198859451434]
  We propose a novel object-level mapping system that can simultaneously segment, track, and reconstruct objects in dynamic scenes. It can further predict and complete their full geometries by conditioning on reconstructions from depth inputs and a category-level shape prior. We evaluate its effectiveness by quantitatively and qualitatively testing it in both synthetic and real-world sequences.
  arXiv  Detail & Related papers  (2022-08-09T22:56:33Z)
• Disentangling Geometric Deformation Spaces in Generative Latent Shape Models [5.582957809895198]
  A complete representation of 3D objects requires characterizing the space of deformations in an interpretable manner. We improve on a prior generative model of disentanglement for 3D shapes, wherein the space of object geometry is factorized into rigid orientation, non-rigid pose, and intrinsic shape. The resulting model can be trained from raw 3D shapes, without correspondences, labels, or even rigid alignment.
  arXiv  Detail & Related papers  (2021-02-27T06:54:31Z)
• Category Level Object Pose Estimation via Neural Analysis-by-Synthesis [64.14028598360741]
  In this paper we combine a gradient-based fitting procedure with a parametric neural image synthesis module. The image synthesis network is designed to efficiently span the pose configuration space. We experimentally show that the method can recover orientation of objects with high accuracy from 2D images alone.
  arXiv  Detail & Related papers  (2020-08-18T20:30:47Z)
• Closed-Form Factorization of Latent Semantics in GANs [65.42778970898534]
  A rich set of interpretable dimensions has been shown to emerge in the latent space of the Generative Adversarial Networks (GANs) trained for synthesizing images. In this work, we examine the internal representation learned by GANs to reveal the underlying variation factors in an unsupervised manner. We propose a closed-form factorization algorithm for latent semantic discovery by directly decomposing the pre-trained weights.
  arXiv  Detail & Related papers  (2020-07-13T18:05:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.