Beyond Prompts: Unconditional 3D Inversion for Out-of-Distribution Shapes

• URL: http://arxiv.org/abs/2604.14914v1
• Date: Thu, 16 Apr 2026 11:55:29 GMT
• Title: Beyond Prompts: Unconditional 3D Inversion for Out-of-Distribution Shapes
• Abstract summary: We demonstrate that for state-of-the-art native text-to-3D generative models, this assumption often collapses.<n>We identify a critical failure mode where generation trajectories are drawn into latent sink traps''<n>In these regimes, changes to the input text fail to alter internal representations in a way that alters the output geometry.<n>Our approach addresses the limitations of current 3D pipelines and enables high-fidelity semantic manipulation of out-of-distribution 3D shapes.
• Score: 29.504851826197996
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Text-driven inversion of generative models is a core paradigm for manipulating 2D or 3D content, unlocking numerous applications such as text-based editing, style transfer, or inverse problems. However, it relies on the assumption that generative models remain sensitive to natural language prompts. We demonstrate that for state-of-the-art native text-to-3D generative models, this assumption often collapses. We identify a critical failure mode where generation trajectories are drawn into latent ``sink traps'': regions where the model becomes insensitive to prompt modifications. In these regimes, changes to the input text fail to alter internal representations in a way that alters the output geometry. Crucially, we observe that this is not a limitation of the model's \textit{geometric} expressivity; the same generative models possess the ability to produce a vast diversity of shapes but, as we demonstrate, become insensitive to out-of-distribution \textit{text} guidance. We investigate this behavior by analyzing the sampling trajectories of the generative model, and find that complex geometries can still be represented and produced by leveraging the model's unconditional generative prior. This leads to a more robust framework for text-based 3D shape editing that bypasses latent sinks by decoupling a model's geometric representation power from its linguistic sensitivity. Our approach addresses the limitations of current 3D pipelines and enables high-fidelity semantic manipulation of out-of-distribution 3D shapes. Project webpage: https://daidedou.sorpi.fr/publication/beyondprompts

Related papers

• SIC3D: Style Image Conditioned Text-to-3D Gaussian Splatting Generation [5.068916657758997]
  SIC3D is a controllable image-conditioned text-to-3D generation pipeline with 3D Gaussian Splatting (3DGS)<n>The first stage generates the 3D object content from text with a text-to-3DGS generation model.<n>The second stage transfers style from a reference image to the 3DGS.
  arXiv  Detail & Related papers  (2026-04-09T20:50:49Z)
• Image-Guided Geometric Stylization of 3D Meshes [23.354227809076345]
  We propose a geometric stylization framework that deforms a 3D mesh, allowing it to express the style of an image.<n>Our coarse-to-fine stylization pipeline can drastically deform the input 3D model to express a diverse range of geometric variations.<n>We also propose an approximate VAE encoder that provides efficient and reliable gradients from mesh renderings.
  arXiv  Detail & Related papers  (2026-04-09T04:44:42Z)
• Know3D: Prompting 3D Generation with Knowledge from Vision-Language Models [39.48274812723658]
  Know3D is a novel framework that incorporates rich knowledge from large language models into 3D generative processes.<n>We successfully bridge the gap between abstract instructions and the geometric reconstruction of unobserved regions.
  arXiv  Detail & Related papers  (2026-03-24T04:18:29Z)
• Text-Image Conditioned 3D Generation [71.98375600100856]
  TIGON is a minimalist dual-branch baseline with separate image- and text-conditioned backbones and lightweight cross-modal fusion.<n>Our diagnostic study shows that even simple late fusion of text- and image-conditioned predictions outperforms single-modality models.<n>Extensive experiments show that text-image conditioning consistently improves over single-modality methods.
  arXiv  Detail & Related papers  (2026-03-22T15:36:16Z)
• StdGEN++: A Comprehensive System for Semantic-Decomposed 3D Character Generation [57.06461272772509]
  StdGEN++ is a novel and comprehensive system for generating high-fidelity, semantically decomposed 3D characters from diverse inputs.<n>It achieves state-of-the-art performance, significantly outperforming existing methods in geometric accuracy and semantic disentanglement.<n>The resulting structural independence unlocks advanced downstream capabilities, including non-destructive editing, physics-compliant animation, and gaze tracking.
  arXiv  Detail & Related papers  (2026-01-12T15:41:27Z)
• Shape from Semantics: 3D Shape Generation from Multi-View Semantics [30.969299308083723]
  Existing 3D reconstruction methods utilize guidances such as 2D images, 3D point clouds, shape contours and single semantics to recover the 3D surface.<n>We propose a novel 3D modeling task called Shape from Semantics'', which aims to create 3D models whose geometry and appearance are consistent with the given text semantics when viewed from different views.
  arXiv  Detail & Related papers  (2025-02-01T07:51:59Z)
• GaussianAnything: Interactive Point Cloud Flow Matching For 3D Object Generation [75.39457097832113]
  This paper introduces a novel 3D generation framework, offering scalable, high-quality 3D generation with an interactive Point Cloud-structured Latent space.<n>Our framework employs a Variational Autoencoder with multi-view posed RGB-D(epth)-N(ormal) renderings as input, using a unique latent space design that preserves 3D shape information.<n>The proposed method, GaussianAnything, supports multi-modal conditional 3D generation, allowing for point cloud, caption, and single image inputs.
  arXiv  Detail & Related papers  (2024-11-12T18:59:32Z)
• 3D Semantic Subspace Traverser: Empowering 3D Generative Model with Shape Editing Capability [13.041974495083197]
  Previous studies on 3D shape generation have focused on shape quality and structure, without or less considering the importance of semantic information. We propose a novel semantic generative model named 3D Semantic Subspace Traverser. Our method can produce plausible shapes with complex structures and enable the editing of semantic attributes.
  arXiv  Detail & Related papers  (2023-07-26T09:04:27Z)
• Controllable 3D Generative Adversarial Face Model via Disentangling Shape and Appearance [63.13801759915835]
  3D face modeling has been an active area of research in computer vision and computer graphics. This paper proposes a new 3D face generative model that can decouple identity and expression.
  arXiv  Detail & Related papers  (2022-08-30T13:40:48Z)
• Disentangled3D: Learning a 3D Generative Model with Disentangled Geometry and Appearance from Monocular Images [94.49117671450531]
  State-of-the-art 3D generative models are GANs which use neural 3D volumetric representations for synthesis. In this paper, we design a 3D GAN which can learn a disentangled model of objects, just from monocular observations.
  arXiv  Detail & Related papers  (2022-03-29T22:03:18Z)
• Identity-Expression Ambiguity in 3D Morphable Face Models [5.38250259923059]
  We show that non-orthogonality of the variation in identity and expression can cause identity-expression ambiguity in 3D Morphable Models. We demonstrate this effect with 3D shapes directly as well as through an inverse rendering task.
  arXiv  Detail & Related papers  (2021-09-29T06:11:43Z)
• SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes [117.76767853430243]
  We introduce SNARF, which combines the advantages of linear blend skinning for polygonal meshes with neural implicit surfaces. We propose a forward skinning model that finds all canonical correspondences of any deformed point using iterative root finding. Compared to state-of-the-art neural implicit representations, our approach generalizes better to unseen poses while preserving accuracy.
  arXiv  Detail & Related papers  (2021-04-08T17:54:59Z)
• Deep Implicit Templates for 3D Shape Representation [70.9789507686618]
  We propose a new 3D shape representation that supports explicit correspondence reasoning in deep implicit representations. Our key idea is to formulate DIFs as conditional deformations of a template implicit function. We show that our method can not only learn a common implicit template for a collection of shapes, but also establish dense correspondences across all the shapes simultaneously without any supervision.
  arXiv  Detail & Related papers  (2020-11-30T06:01:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.