UniRecGen: Unifying Multi-View 3D Reconstruction and Generation

• URL: http://arxiv.org/abs/2604.01479v2
• Date: Fri, 03 Apr 2026 02:02:55 GMT
• Title: UniRecGen: Unifying Multi-View 3D Reconstruction and Generation
• Abstract summary: We present UniRecGen, a unified framework that integrates feed-forward and diffusion-based generation.<n>We employ disentangled cooperative learning, which maintains stable training while enabling seamless collaboration during inference.<n> Experimental results demonstrate that UniRecGen achieves superior fidelity and robustness, outperforming existing methods.
• Score: 39.26563127393024
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sparse-view 3D modeling represents a fundamental tension between reconstruction fidelity and generative plausibility. While feed-forward reconstruction excels in efficiency and input alignment, it often lacks the global priors needed for structural completeness. Conversely, diffusion-based generation provides rich geometric details but struggles with multi-view consistency. We present UniRecGen, a unified framework that integrates these two paradigms into a single cooperative system. To overcome inherent conflicts in coordinate spaces, 3D representations, and training objectives, we align both models within a shared canonical space. We employ disentangled cooperative learning, which maintains stable training while enabling seamless collaboration during inference. Specifically, the reconstruction module is adapted to provide canonical geometric anchors, while the diffusion generator leverages latent-augmented conditioning to refine and complete the geometric structure. Experimental results demonstrate that UniRecGen achieves superior fidelity and robustness, outperforming existing methods in creating complete and consistent 3D models from sparse observations. Code is available at https://github.com/zsh523/UniRecGen.

Related papers

• Stable and Scalable Bundle Adjustment of Holistic 3D Structures [99.03818732955783]
  We present a new version of Bundle Adjustment (BA) for 3D computer vision.<n>BA is developed to jointly optimize cameras, poses and 3D points.<n>By group-induced measurements and cross-feature reprojection, we preserve the sparsity of classical point-only 3D regularization.
  arXiv  Detail & Related papers  (2026-09-03T16:08:03Z)
• Pano3D: Unified 3D Reconstruction and Panoptic Segmentation [68.18993232814061]
  We introduce an approach that performs 3D reconstruction and 3D panoptic segmentation in a unified framework.<n>We build on existing 3D reconstruction models and augment them with a set-based mask decoder.<n>Our method achieves state-of-the-art performance in 3D panoptic segmentation across ScanNet, ScanNet200, and ScanNet++ datasets.
  arXiv  Detail & Related papers  (2026-06-12T09:43:13Z)
• Mind the Gap: Geometrically Accurate Generative Reconstruction from Disjoint Views [0.8435195043404353]
  We introduce Generative Reconstruction from Disjoint Views as a new paradigm.<n>We propose GLADOS, a general, modular framework that operates through three stages.<n>As an architectureagnostic framework, GLADOS enables seamless integration of future advances in generation, reconstruction, and inpainting.
  arXiv  Detail & Related papers  (2026-05-08T10:24:23Z)
• Interact3D: Compositional 3D Generation of Interactive Objects [31.12099147294145]
  We present a novel framework designed to generate physically plausible interacting 3D compositional objects.<n>Our approach first leverages advanced generative priors to curate high-quality individual assets.<n>To physically compose these assets, we then introduce a robust two-stage composition pipeline.
  arXiv  Detail & Related papers  (2026-03-17T03:21:06Z)
• RAYNOVA: Scale-Temporal Autoregressive World Modeling in Ray Space [51.441415833480505]
  RAYNOVA is a multiview world model for driving scenarios that employs a dual-causal autoregressive framework.<n>It constructs an isotropic-temporal representation across views, frames, and scales based on relative Plcker-ray positional encoding.
  arXiv  Detail & Related papers  (2026-02-24T08:41:40Z)
• Joint Geometry-Appearance Human Reconstruction in a Unified Latent Space via Bridge Diffusion [57.09673862519791]
  This paper introduces textbfJGA-LBD, a novel framework that unifies the modeling of geometry and appearance into a joint latent representation.<n> Experiments demonstrate that JGA-LBD outperforms current state-of-the-art approaches in terms of both geometry fidelity and appearance quality.
  arXiv  Detail & Related papers  (2026-01-01T12:48:56Z)
• ReconViaGen: Towards Accurate Multi-view 3D Object Reconstruction via Generation [28.308731720451053]
  We propose ReconViaGen to integrate reconstruction priors into the generative framework.<n>Our experiments demonstrate that our ReconViaGen can reconstruct complete and accurate 3D models consistent with input views in both global structure and local details.
  arXiv  Detail & Related papers  (2025-10-27T13:15:06Z)
• Seeing 3D Through 2D Lenses: 3D Few-Shot Class-Incremental Learning via Cross-Modal Geometric Rectification [59.17489431187807]
  We propose a framework that enhances 3D geometric fidelity by leveraging CLIP's hierarchical spatial semantics.<n>Our method significantly improves 3D few-shot class-incremental learning, achieving superior geometric coherence and robustness to texture bias.
  arXiv  Detail & Related papers  (2025-09-18T13:45:08Z)
• H3R: Hybrid Multi-view Correspondence for Generalizable 3D Reconstruction [39.22287224290769]
  H3R is a hybrid framework that integrates latent fusion with attention-based feature aggregation.<n>By integrating both paradigms, our approach enhances generalization while converging 2$times$ faster than existing methods.<n>Our method supports variable-number and high-resolution input views while demonstrating robust cross-dataset generalization.
  arXiv  Detail & Related papers  (2025-08-05T05:56:30Z)
• From One to More: Contextual Part Latents for 3D Generation [38.36190651170286]
  CoPart is a part-aware diffusion framework that decomposes 3D objects into contextual part latents for coherent multi-part generation.<n>We construct a novel 3D part dataset derived from articulated mesh segmentation and human-verified annotations.<n>Experiments demonstrate CoPart's superior capabilities in part-level editing, object generation, and scene composition with unprecedented controllability.
  arXiv  Detail & Related papers  (2025-07-11T17:33:18Z)
• GTR: Gaussian Splatting Tracking and Reconstruction of Unknown Objects Based on Appearance and Geometric Complexity [49.31257173003408]
  We present a novel method for 6-DoF object tracking and high-quality 3D reconstruction from monocular RGBD video.<n>Our approach demonstrates strong capabilities in recovering high-fidelity object meshes, setting a new standard for single-sensor 3D reconstruction in open-world environments.
  arXiv  Detail & Related papers  (2025-05-17T08:46:29Z)
• PRISM: Probabilistic Representation for Integrated Shape Modeling and Generation [79.46526296655776]
  PRISM is a novel approach for 3D shape generation that integrates categorical diffusion models with Statistical Shape Models (SSM) and Gaussian Mixture Models (GMM)<n>Our method employs compositional SSMs to capture part-level geometric variations and uses GMM to represent part semantics in a continuous space.<n>Our approach significantly outperforms previous methods in both quality and controllability of part-level operations.
  arXiv  Detail & Related papers  (2025-04-06T11:48:08Z)
• Evolving High-Quality Rendering and Reconstruction in a Unified Framework with Contribution-Adaptive Regularization [27.509109317973817]
  3D Gaussian Splatting (3DGS) has garnered significant attention for its high-quality rendering and fast inference speed.<n>Previous methods primarily focus on geometry regularization, with common approaches including primitive-based and dual-model frameworks.<n>We propose CarGS, a unified model leveraging-adaptive regularization to achieve simultaneous, high-quality surface reconstruction.
  arXiv  Detail & Related papers  (2025-03-02T12:51:38Z)
• JADE: Joint-aware Latent Diffusion for 3D Human Generative Modeling [62.77347895550087]
  We introduce JADE, a generative framework that learns the variations of human shapes with fined-grained control.<n>Our key insight is a joint-aware latent representation that decomposes human bodies into skeleton structures.<n>To generate coherent and plausible human shapes under our proposed decomposition, we also present a cascaded pipeline.
  arXiv  Detail & Related papers  (2024-12-29T14:18:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.