Structural Energy Guidance for View-Consistent Text-to-3D Generation

• URL: http://arxiv.org/abs/2605.19876v1
• Date: Tue, 19 May 2026 14:08:52 GMT
• Title: Structural Energy Guidance for View-Consistent Text-to-3D Generation
• Abstract summary: This work identifies viewpoint bias in 2D diffusion priors as the main cause and proposes Structural Energy-Guided Sampling (SEGS) to improve multi-view consistency.<n>SEGS constructs a structural energy in the PCA subspace of U-Net features and injects its gradient into the denoising process.<n> Experiments show that SEGS reduces the Janus Rate by about 10% on average and improves View-CS scores across multiple baselines, including DreamFusion, Magic3D, and LucidDreamer.
• Score: 18.973527029488746
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text-to-3D generation based on diffusion models often suffers from the Janus problem, leading to inconsistent geometry across viewpoints. This work identifies viewpoint bias in 2D diffusion priors as the main cause and proposes Structural Energy-Guided Sampling (SEGS), a training-free and plug-and-play framework to improve multi-view consistency. SEGS constructs a structural energy in the PCA subspace of U-Net features and injects its gradient into the denoising process. It can be easily integrated into SDS/VSD pipelines without retraining. Experiments show that SEGS reduces the Janus Rate by about 10% on average and improves View-CS scores across multiple baselines, including DreamFusion, Magic3D, and LucidDreamer. This method effectively alleviates viewpoint artifacts while preserving appearance fidelity, providing a flexible solution for high-quality text-to-3D content generation.

Related papers

• TV-SGS: Gaussian Splatting with Geometric Information Propagation via Tensor Voting under sparse views [5.274411980510585]
  We introduce a new class of 3D losses that do not rely on rendering and can be combined with essentially all losses previously reported in the literature.<n>Our method is easy to integrate with a diverse set of backbones, and our experiments on the DTU and Tanks-and-Temples datasets demonstrate that TV-SGS improves the geometry of the outputs compared to the backbone.
  arXiv  Detail & Related papers  (2026-09-07T16:40:51Z)
• DualDiff3D: Dual Structure-Appearance Diffusion Priors for Reliability-Enhanced 3D Gaussian Splatting [71.316050612276]
  3D reconstruction and novel-view synthesis often lead to poor reconstruction quality and artifacts in rendered novel views.<n>Recent efforts attempt to utilize powerful diffusion priors, yet they typically process rendered and reference views along an additional dimension in a single network.<n>We propose DualDiff, a novel pipeline that leverages dual diffusion priors with a Structure-Appearance Attention (SAA) module.<n>Our approach outperforms state-of-the-art methods even in the inference-only setting, with further performance gains achievable through training.
  arXiv  Detail & Related papers  (2026-09-01T16:45:53Z)
• CGGS: Consistency-Augmented Geometric Gaussian Splatting for Ego-centric 3D Scene Generation [14.481547285312928]
  CGGS aims to enhance 3D-content-awareness and address geometric distortions in ego-centric scene generation.<n>Ego-centric Generator is proposed by fine-tuning a Multi-View Latent Diffusion Model with consistency-augmented loss.<n> layout Decorator leverages optical flow and point-track correspondence to estimate depth.<n> Geometric Refiner is proposed to enhance 3D Gaussian reconstruction via an entropy-based Mutual Information Depth Loss.
  arXiv  Detail & Related papers  (2026-07-04T11:06:41Z)
• OneWorld: Taming Scene Generation with 3D Unified Representation Autoencoder [90.8453349494245]
  We present OneWorld, a framework that performs diffusion directly within a coherent 3D representation space.<n>OneWorld generates high-quality 3D scenes with superior cross-view consistency compared to state-of-the-art 2D-based methods.
  arXiv  Detail & Related papers  (2026-03-17T03:43:37Z)
• Scalable Adaptation of 3D Geometric Foundation Models via Weak Supervision from Internet Video [76.32954467706581]
  We propose SAGE, a framework for Scalable Adaptation of GEometric foundation models from raw video streams.<n>We use a hierarchical mining pipeline to transform videos into training trajectories and hybrid supervision.<n>Experiments show that SAGE significantly enhances zero-shot generalization, reducing Chamfer Distance by 20-42% on unseen benchmarks.
  arXiv  Detail & Related papers  (2026-02-08T09:53:21Z)
• One-Shot Refiner: Boosting Feed-forward Novel View Synthesis via One-Step Diffusion [57.824020826432815]
  We present a novel framework for high-fidelity novel view synthesis (NVS) from sparse images.<n>We design a Dual-Domain Detail Perception Module, which enables handling high-resolution images without being limited by the ViT backbone.<n>We develop a feature-guided diffusion network, which can preserve high-frequency details during the restoration process.
  arXiv  Detail & Related papers  (2026-01-20T17:11:55Z)
• FaithFusion: Harmonizing Reconstruction and Generation via Pixel-wise Information Gain [17.131480990824397]
  We introduce a 3DGS-diffusion framework driven by pixel-wise Information Gain (EIG)<n>EIG acts as a unified policy for coherent controllable-text synthesis.<n>Experiments on a dataset demonstrate that our approach attains SOTA across NTA-oU, NTLI-oU, and FID, maintaining an FID of 107.47 even at 6 meters lane shift.
  arXiv  Detail & Related papers  (2025-11-26T06:58:57Z)
• LSS3D: Learnable Spatial Shifting for Consistent and High-Quality 3D Generation from Single-Image [12.683334603689502]
  We propose a high-quality image-to-3D approach, named LSS3D, with learnable spatial shifting to handle the multiview inconsistencies and non-frontal input view.<n>Specifically, we assign learnable spatial shifting parameters to each view, and adjust each view towards a spatially consistent target, guided by the reconstructed mesh.<n>Our method consistently achieves leading results in both geometric and texture evaluation metrics across more flexible input viewpoints.
  arXiv  Detail & Related papers  (2025-11-15T13:14:25Z)
• Structural Energy-Guided Sampling for View-Consistent Text-to-3D [18.973527029488746]
  Text-to-3D generation often suffers from the Janus problem, where objects collapse into duplicated or distorted geometry from other angles.<n>We propose Structural Energy-Guided Sampling (SEGS), a training-free, plug-and-play framework that enforces multi-view consistency entirely at sampling time.
  arXiv  Detail & Related papers  (2025-08-23T06:26:04Z)
• Decompositional Neural Scene Reconstruction with Generative Diffusion Prior [64.71091831762214]
  Decompositional reconstruction of 3D scenes, with complete shapes and detailed texture, is intriguing for downstream applications.<n>Recent approaches incorporate semantic or geometric regularization to address this issue, but they suffer significant degradation in underconstrained areas.<n>We propose DP-Recon, which employs diffusion priors in the form of Score Distillation Sampling (SDS) to optimize the neural representation of each individual object under novel views.
  arXiv  Detail & Related papers  (2025-03-19T02:11:31Z)
• Improving Viewpoint Consistency in 3D Generation via Structure Feature and CLIP Guidance [17.867010229092735]
  Current methods often suffer from geometric inconsistencies, commonly referred to as the Janus Problem.<n>This paper identifies the root cause of the Janus Problem: viewpoint generation bias in diffusion models.<n>We propose a tuning-free approach called the Attention and CLIP Guidance (ACG) mechanism.
  arXiv  Detail & Related papers  (2024-12-03T09:05:32Z)
• GaussianAnything: Interactive Point Cloud Flow Matching For 3D Object Generation [75.39457097832113]
  This paper introduces a novel 3D generation framework, offering scalable, high-quality 3D generation with an interactive Point Cloud-structured Latent space.<n>Our framework employs a Variational Autoencoder with multi-view posed RGB-D(epth)-N(ormal) renderings as input, using a unique latent space design that preserves 3D shape information.<n>The proposed method, GaussianAnything, supports multi-modal conditional 3D generation, allowing for point cloud, caption, and single image inputs.
  arXiv  Detail & Related papers  (2024-11-12T18:59:32Z)
• Sculpt3D: Multi-View Consistent Text-to-3D Generation with Sparse 3D Prior [57.986512832738704]
  We present a new framework Sculpt3D that equips the current pipeline with explicit injection of 3D priors from retrieved reference objects without re-training the 2D diffusion model. Specifically, we demonstrate that high-quality and diverse 3D geometry can be guaranteed by keypoints supervision through a sparse ray sampling approach. These two decoupled designs effectively harness 3D information from reference objects to generate 3D objects while preserving the generation quality of the 2D diffusion model.
  arXiv  Detail & Related papers  (2024-03-14T07:39:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.