SegmentDreamer: Towards High-fidelity Text-to-3D Synthesis with Segmented Consistency Trajectory Distillation

• URL: http://arxiv.org/abs/2507.05256v2
• Date: Sun, 03 Aug 2025 13:51:56 GMT
• Title: SegmentDreamer: Towards High-fidelity Text-to-3D Synthesis with Segmented Consistency Trajectory Distillation
• Authors: Jiahao Zhu, Zixuan Chen, Guangcong Wang, Xiaohua Xie, Yi Zhou,
• Abstract summary: Recent advancements in text-to-3D generation improve the visual quality of Score Distillation Sampling (SDS) and its variants.<n>Due to the imbalance between self-consistency and cross-consistency, CD-based methods inherently suffer from improper conditional guidance.<n>We present SegmentDreamer, a novel framework designed to fully unleash the potential of consistency models for high-fidelity text-to-3D generation.
• Score: 37.329698607074114
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Recent advancements in text-to-3D generation improve the visual quality of Score Distillation Sampling (SDS) and its variants by directly connecting Consistency Distillation (CD) to score distillation. However, due to the imbalance between self-consistency and cross-consistency, these CD-based methods inherently suffer from improper conditional guidance, leading to sub-optimal generation results. To address this issue, we present SegmentDreamer, a novel framework designed to fully unleash the potential of consistency models for high-fidelity text-to-3D generation. Specifically, we reformulate SDS through the proposed Segmented Consistency Trajectory Distillation (SCTD), effectively mitigating the imbalance issues by explicitly defining the relationship between self- and cross-consistency. Moreover, SCTD partitions the Probability Flow Ordinary Differential Equation (PF-ODE) trajectory into multiple sub-trajectories and ensures consistency within each segment, which can theoretically provide a significantly tighter upper bound on distillation error. Additionally, we propose a distillation pipeline for a more swift and stable generation. Extensive experiments demonstrate that our SegmentDreamer outperforms state-of-the-art methods in visual quality, enabling high-fidelity 3D asset creation through 3D Gaussian Splatting (3DGS).

Related papers

• Dive3D: Diverse Distillation-based Text-to-3D Generation via Score Implicit Matching [14.267619174518106]
  We introduce Dive3D, a novel text-to-3D generation framework that replaces KL-based objectives with Score Implicit Matching (SIM) loss.<n>We validate Dive3D across various 2D-to-3D prompts and find that it consistently outperforms prior methods in qualitative assessments.<n> Dive3D also achieves strong results on quantitative metrics, including text-asset alignment, 3D plausibility, text-geometry consistency, texture quality, and geometric detail.
  arXiv  Detail & Related papers  (2025-06-16T15:21:30Z)
• Acc3D: Accelerating Single Image to 3D Diffusion Models via Edge Consistency Guided Score Distillation [49.202383675543466]
  We present Acc3D to tackle the challenge of accelerating the diffusion process to generate 3D models from single images.<n>To derive high-quality reconstructions through few-step inferences, we emphasize the critical issue of regularizing the learning of score function in states of random noise.
  arXiv  Detail & Related papers  (2025-03-20T09:18:10Z)
• Consistent Flow Distillation for Text-to-3D Generation [14.150490171643034]
  Score Distillation Sampling (SDS) has made significant strides in distilling image-generative models for 3D generation.<n>However, its maximum-likelihood-seeking behavior often leads to degraded visual quality and diversity, limiting its effectiveness in 3D applications.<n>We propose Consistent Flow Distillation (CFD), which addresses these limitations.
  arXiv  Detail & Related papers  (2025-01-09T18:56:05Z)
• Diverse Score Distillation [27.790458964072823]
  We propose a score formulation that guides the optimization to follow generation paths defined by random initial seeds.<n>We showcase the applications of our Diverse Score Distillation' (DSD) formulation across tasks such as 2D optimization, text-based 3D inference, and single-view reconstruction.
  arXiv  Detail & Related papers  (2024-12-09T18:59:02Z)
• VividDreamer: Invariant Score Distillation For Hyper-Realistic Text-to-3D Generation [33.05759961083337]
  This paper presents Invariant Score Distillation (ISD), a novel method for high-fidelity text-to-3D generation. ISD aims to tackle the over-saturation and over-smoothing problems in Score Distillation Sampling (SDS)
  arXiv  Detail & Related papers  (2024-07-13T09:33:16Z)
• Trajectory Consistency Distillation: Improved Latent Consistency Distillation by Semi-Linear Consistency Function with Trajectory Mapping [75.72212215739746]
  Trajectory Consistency Distillation (TCD) encompasses trajectory consistency function and strategic sampling. TCD significantly enhances image quality at low NFEs but also yields more detailed results compared to the teacher model.
  arXiv  Detail & Related papers  (2024-02-29T13:44:14Z)
• SteinDreamer: Variance Reduction for Text-to-3D Score Distillation via Stein Identity [70.32101198891465]
  We show that gradient estimation in score distillation is inherent to high variance. We propose a more general solution to reduce variance for score distillation, termed Stein Score Distillation (SSD) We demonstrate that SteinDreamer achieves faster convergence than existing methods due to more stable gradient updates.
  arXiv  Detail & Related papers  (2023-12-31T23:04:25Z)
• Taming Mode Collapse in Score Distillation for Text-to-3D Generation [70.32101198891465]
  "Janus" artifact is a problem in text-to-3D generation where the generated objects fake each view with multiple front faces. We propose a new update rule for 3D score distillation, dubbed Entropic Score Distillation ( ESD) Although embarrassingly straightforward, our experiments successfully demonstrate that ESD can be an effective treatment for Janus artifacts in score distillation.
  arXiv  Detail & Related papers  (2023-12-31T22:47:06Z)
• Stable Score Distillation for High-Quality 3D Generation [21.28421571320286]
  We decompose Score Distillation Sampling (SDS) as a combination of three functional components, namely mode-seeking, mode-disengaging and variance-reducing terms. We show that problems such as over-smoothness and implausibility result from the intrinsic deficiency of the first two terms. We propose a simple yet effective approach named Stable Score Distillation (SSD) which strategically orchestrates each term for high-quality 3D generation.
  arXiv  Detail & Related papers  (2023-12-14T19:18:38Z)
• Learn to Optimize Denoising Scores for 3D Generation: A Unified and Improved Diffusion Prior on NeRF and 3D Gaussian Splatting [60.393072253444934]
  We propose a unified framework aimed at enhancing the diffusion priors for 3D generation tasks. We identify a divergence between the diffusion priors and the training procedures of diffusion models that substantially impairs the quality of 3D generation.
  arXiv  Detail & Related papers  (2023-12-08T03:55:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.