DecompDreamer: Advancing Structured 3D Asset Generation with Multi-Object Decomposition and Gaussian Splatting

• URL: http://arxiv.org/abs/2503.11981v1
• Date: Sat, 15 Mar 2025 03:37:25 GMT
• Title: DecompDreamer: Advancing Structured 3D Asset Generation with Multi-Object Decomposition and Gaussian Splatting
• Abstract summary: DecompDreamer is a training routine designed to generate high-quality 3D compositions.<n>It decomposes scenes into structured components and their relationships.<n>It effectively generates intricate 3D compositions with superior object disentanglement.
• Score: 24.719972380079405
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Text-to-3D generation saw dramatic advances in recent years by leveraging Text-to-Image models. However, most existing techniques struggle with compositional prompts, which describe multiple objects and their spatial relationships. They often fail to capture fine-grained inter-object interactions. We introduce DecompDreamer, a Gaussian splatting-based training routine designed to generate high-quality 3D compositions from such complex prompts. DecompDreamer leverages Vision-Language Models (VLMs) to decompose scenes into structured components and their relationships. We propose a progressive optimization strategy that first prioritizes joint relationship modeling before gradually shifting toward targeted object refinement. Our qualitative and quantitative evaluations against state-of-the-art text-to-3D models demonstrate that DecompDreamer effectively generates intricate 3D compositions with superior object disentanglement, offering enhanced control and flexibility in 3D generation. Project page : https://decompdreamer3d.github.io

Related papers

• Stable and Scalable Bundle Adjustment of Holistic 3D Structures [99.03818732955783]
  We present a new version of Bundle Adjustment (BA) for 3D computer vision.<n>BA is developed to jointly optimize cameras, poses and 3D points.<n>By group-induced measurements and cross-feature reprojection, we preserve the sparsity of classical point-only 3D regularization.
  arXiv  Detail & Related papers  (2026-09-03T16:08:03Z)
• Revitalizing the Beginning: Avoiding Storage Dependency for Model Merging in Continual Learning [57.10440766103372]
  Trajectory Regularized Merging (TRM) is a framework that reformulates the merging phase as an optimization process within an augmented trajectory subspace.<n>Our framework integrates three synergistic objectives including task alignment, prediction consistency, and gradient responsiveness to concurrently preserve merged model's historical stability and re-activate optimization dynamics.
  arXiv  Detail & Related papers  (2026-05-08T14:07:32Z)
• Beyond the Academic Monoculture: A Unified Framework and Industrial Perspective for Attributed Graph Clustering [75.50670592447219]
  Attributed Graph Clustering (AGC) is a fundamental unsupervised task that partitions nodes into cohesive groups by jointly modeling structural topology and node attributes.<n>This survey provides a comprehensive, industrially grounded review of AGC from three complementary perspectives.
  arXiv  Detail & Related papers  (2026-03-21T14:15:34Z)
• S-VGGT: Structure-Aware Subscene Decomposition for Scalable 3D Foundation Models [15.408916900664783]
  Feed-forward 3D foundation models face a key challenge: the quadratic computational cost introduced by global attention.<n>We introduce textbfS-VGGT, a novel approach that addresses redundancy at the structural frame level.<n>S-VGGT is entirely to token-level acceleration methods, allowing the two to be seamlessly combined for compounded speedups.
  arXiv  Detail & Related papers  (2026-03-18T11:42:55Z)
• Interact3D: Compositional 3D Generation of Interactive Objects [31.12099147294145]
  We present a novel framework designed to generate physically plausible interacting 3D compositional objects.<n>Our approach first leverages advanced generative priors to curate high-quality individual assets.<n>To physically compose these assets, we then introduce a robust two-stage composition pipeline.
  arXiv  Detail & Related papers  (2026-03-17T03:21:06Z)
• StructAlign: Structured Cross-Modal Alignment for Continual Text-to-Video Retrieval [75.28673512571449]
  A key challenge in Continual Text-to-Video Retrieval is feature drift.<n>We propose StructAlign, a structured cross-modal alignment method for CTVR.<n>Our method consistently outperforms state-of-the-art continual retrieval approaches.
  arXiv  Detail & Related papers  (2026-01-28T13:34:44Z)
• Copy-Trasform-Paste: Zero-Shot Object-Object Alignment Guided by Vision-Language and Geometric Constraints [12.704390013489054]
  We study zero-shot 3D alignment of two given meshes, using a text prompt describing their relation.<n>We optimize the relative pose at test time, updating translation, rotation, and isotropic scale with CLIP-driven gradients.<n>Our method outperforms all alternatives, yielding semantically faithful and physically plausible alignments.
  arXiv  Detail & Related papers  (2026-01-20T18:12:55Z)
• RMLer: Synthesizing Novel Objects across Diverse Categories via Reinforcement Mixing Learning [16.682831359982064]
  Reinforcement Mixing Learning (RMLer) is a framework that formulates cross-category concept fusion.<n>Our work provides a robust framework for generating novel visual concepts, with promising applications in film, gaming, and design.
  arXiv  Detail & Related papers  (2025-12-22T11:44:32Z)
• Beyond Confidence: Adaptive and Coherent Decoding for Diffusion Language Models [64.92045568376705]
  Coherent Contextual Decoding (CCD) is a novel inference framework built upon two core innovations.<n>CCD employs a trajectory rectification mechanism that leverages historical context to enhance sequence coherence.<n>Instead of rigid allocations based on diffusion steps, we introduce an adaptive sampling strategy that dynamically adjusts the unmasking budget for each step.
  arXiv  Detail & Related papers  (2025-11-26T09:49:48Z)
• HiCoGen: Hierarchical Compositional Text-to-Image Generation in Diffusion Models via Reinforcement Learning [66.99487505369254]
  HiCoGen is built upon a novel Chain of Synthesis paradigm.<n>It decomposes complex prompts into minimal semantic units.<n>It then synthesizes these units iteratively, where the image generated in each step provides crucial visual context for the next.<n>Experiments show our approach significantly outperforms existing methods in both concept coverage and compositional accuracy.
  arXiv  Detail & Related papers  (2025-11-25T06:24:25Z)
• CompGS: Unleashing 2D Compositionality for Compositional Text-to-3D via Dynamically Optimizing 3D Gaussians [97.15119679296954]
  CompGS is a novel generative framework that employs 3D Gaussian Splatting (GS) for efficient, compositional text-to-3D content generation. CompGS can be easily extended to controllable 3D editing, facilitating scene generation.
  arXiv  Detail & Related papers  (2024-10-28T04:35:14Z)
• IterComp: Iterative Composition-Aware Feedback Learning from Model Gallery for Text-to-Image Generation [70.8833857249951]
  IterComp is a novel framework that aggregates composition-aware model preferences from multiple models.<n>We propose an iterative feedback learning method to enhance compositionality in a closed-loop manner.<n>IterComp opens new research avenues in reward feedback learning for diffusion models and compositional generation.
  arXiv  Detail & Related papers  (2024-10-09T17:59:13Z)
• DreamScape: 3D Scene Creation via Gaussian Splatting joint Correlation Modeling [23.06464506261766]
  We present DreamScape, a method for creating highly consistent 3D scenes solely from textual descriptions. Our approach involves a 3D Gaussian Guide for scene representation, consisting of semantic primitives (objects) and their spatial transformations. A progressive scale control is tailored during local object generation, ensuring that objects of different sizes and densities adapt to the scene.
  arXiv  Detail & Related papers  (2024-04-14T12:13:07Z)
• Retrieval-Augmented Score Distillation for Text-to-3D Generation [30.57225047257049]
  We introduce novel framework for retrieval-based quality enhancement in text-to-3D generation. We conduct extensive experiments to demonstrate that ReDream exhibits superior quality with increased geometric consistency.
  arXiv  Detail & Related papers  (2024-02-05T12:50:30Z)
• TeMO: Towards Text-Driven 3D Stylization for Multi-Object Meshes [67.5351491691866]
  We present a novel framework, dubbed TeMO, to parse multi-object 3D scenes and edit their styles. Our method can synthesize high-quality stylized content and outperform the existing methods over a wide range of multi-object 3D meshes.
  arXiv  Detail & Related papers  (2023-12-07T12:10:05Z)
• GraphDreamer: Compositional 3D Scene Synthesis from Scene Graphs [74.98581417902201]
  We propose a novel framework to generate compositional 3D scenes from scene graphs. By exploiting node and edge information in scene graphs, our method makes better use of the pretrained text-to-image diffusion model. We conduct both qualitative and quantitative experiments to validate the effectiveness of GraphDreamer.
  arXiv  Detail & Related papers  (2023-11-30T18:59:58Z)
• CG3D: Compositional Generation for Text-to-3D via Gaussian Splatting [57.14748263512924]
  CG3D is a method for compositionally generating scalable 3D assets. Gamma radiance fields, parameterized to allow for compositions of objects, possess the capability to enable semantically and physically consistent scenes.
  arXiv  Detail & Related papers  (2023-11-29T18:55:38Z)
• IPDreamer: Appearance-Controllable 3D Object Generation with Complex Image Prompts [90.49024750432139]
  We present IPDreamer, a novel method that captures intricate appearance features from complex $textbfI$mage $textbfP$rompts and aligns the synthesized 3D object with these extracted features. Our experiments demonstrate that IPDreamer consistently generates high-quality 3D objects that align with both the textual and complex image prompts.
  arXiv  Detail & Related papers  (2023-10-09T03:11:08Z)
• ATT3D: Amortized Text-to-3D Object Synthesis [78.96673650638365]
  We amortize optimization over text prompts by training on many prompts simultaneously with a unified model, instead of separately. Our framework - Amortized text-to-3D (ATT3D) - enables knowledge-sharing between prompts to generalize to unseen setups and smooths between text for novel assets and simple animations.
  arXiv  Detail & Related papers  (2023-06-06T17:59:10Z)
• Understanding and Constructing Latent Modality Structures in Multi-modal Representation Learning [53.68371566336254]
  We argue that the key to better performance lies in meaningful latent modality structures instead of perfect modality alignment. Specifically, we design 1) a deep feature separation loss for intra-modality regularization; 2) a Brownian-bridge loss for inter-modality regularization; and 3) a geometric consistency loss for both intra- and inter-modality regularization.
  arXiv  Detail & Related papers  (2023-03-10T14:38:49Z)
• Contextual Model Aggregation for Fast and Robust Federated Learning in Edge Computing [88.76112371510999]
  Federated learning is a prime candidate for distributed machine learning at the network edge. Existing algorithms face issues with slow convergence and/or robustness of performance. We propose a contextual aggregation scheme that achieves the optimal context-dependent bound on loss reduction.
  arXiv  Detail & Related papers  (2022-03-23T21:42:31Z)
• Collaborative Multidisciplinary Design Optimization with Neural Networks [1.2691047660244335]
  We show that, in the case of Collaborative Optimization, faster and more reliable convergence can be obtained by solving an interesting instance of binary classification. We propose to train a neural network with an asymmetric loss function, a structure that guarantees Lipshitz continuity, and a regularization towards respecting basic distance function properties.
  arXiv  Detail & Related papers  (2021-06-11T00:03:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.