Towards Realistic and Consistent Orbital Video Generation via 3D Foundation Priors

• URL: http://arxiv.org/abs/2604.12309v1
• Date: Tue, 14 Apr 2026 05:35:46 GMT
• Title: Towards Realistic and Consistent Orbital Video Generation via 3D Foundation Priors
• Abstract summary: We present a novel method for generating orbital videos from a single image of an object.<n>Our method achieves superior visual quality, shape realism and multi-view consistency compared to state-of-the-art methods.
• Score: 61.34273238077091
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a novel method for generating geometrically realistic and consistent orbital videos from a single image of an object. Existing video generation works mostly rely on pixel-wise attention to enforce view consistency across frames. However, such mechanism does not impose sufficient constraints for long-range extrapolation, e.g. rear-view synthesis, in which pixel correspondences to the input image are limited. Consequently, these works often fail to produce results with a plausible and coherent structure. To tackle this issue, we propose to leverage rich shape priors from a 3D foundational generative model as an auxiliary constraint, motivated by its capability of modeling realistic object shape distributions learned from large 3D asset corpora. Specifically, we prompt the video generation with two scales of latent features encoded by the 3D foundation model: (i) a denoised global latent vector as an overall structural guidance, and (ii) a set of latent images projected from volumetric features to provide view-dependent and fine-grained geometry details. In contrast to commonly used 2.5D representations such as depth or normal maps, these compact features can model complete object shapes, and help to improve inference efficiency by avoiding explicit mesh extraction. To achieve effective shape conditioning, we introduce a multi-scale 3D adapter to inject feature tokens to the base video model via cross-attention, which retains its capabilities from general video pretraining and enables a simple and model-agonistic fine-tuning process. Extensive experiments on multiple benchmarks show that our method achieves superior visual quality, shape realism and multi-view consistency compared to state-of-the-art methods, and robustly generalizes to complex camera trajectories and in-the-wild images.

Related papers

• GTA: Advancing Image-to-3D World Generation via Geometry Then Appearance Video Diffusion [29.999238067855245]
  We propose GTA, a novel image-to-3D world generation method following a Geometry-Then-Appearance paradigm.<n>Specifically, given a single input image, GTA adopts a two-stage framework with two dedicated video diffusion models.<n>Extensive experiments have demonstrated that our proposed method consistently outperforms existing approaches in terms of fidelity, visual quality, and geometric accuracy.
  arXiv  Detail & Related papers  (2026-05-13T03:43:02Z)
• TAPESTRY: From Geometry to Appearance via Consistent Turntable Videos [65.99602532894241]
  We introduce TAPESTRY, a framework for generating high-fidelity TTVs conditioned on explicit 3D geometry.<n>We also design a method for downstream reconstruction tasks from the TTV input, featuring a multi-stage pipeline with 3D-Aware Inpainting.<n>Results demonstrate that our method outperforms existing approaches in both video consistency and final reconstruction quality.
  arXiv  Detail & Related papers  (2026-03-18T14:02:09Z)
• OPFormer: Object Pose Estimation leveraging foundation model with geometric encoding [2.1987601456703474]
  We introduce a unified, end-to-end framework that seamlessly integrates object detection and pose estimation.<n>Our system first employs the CNOS detector to localize target objects.<n>For each detection, our novel pose estimation module, OPFormer, infers the precise 6D pose.
  arXiv  Detail & Related papers  (2025-11-16T14:19:52Z)
• Geometry and Perception Guided Gaussians for Multiview-consistent 3D Generation from a Single Image [10.648593818811976]
  Existing approaches often rely on fine-tuning pretrained 2D diffusion models or directly generating 3D information through fast network inference.<n>We present a novel method that seamlessly integrates geometry and perception information without requiring additional model training.<n> Experimental results show that we outperform existing methods on novel view synthesis and 3D reconstruction, demonstrating robust and consistent 3D object generation.
  arXiv  Detail & Related papers  (2025-06-26T11:22:06Z)
• HORT: Monocular Hand-held Objects Reconstruction with Transformers [61.36376511119355]
  Reconstructing hand-held objects in 3D from monocular images is a significant challenge in computer vision.<n>We propose a transformer-based model to efficiently reconstruct dense 3D point clouds of hand-held objects.<n>Our method achieves state-of-the-art accuracy with much faster inference speed, while generalizing well to in-the-wild images.
  arXiv  Detail & Related papers  (2025-03-27T09:45:09Z)
• CDI3D: Cross-guided Dense-view Interpolation for 3D Reconstruction [25.468907201804093]
  Large Reconstruction Models (LRMs) have shown great promise in leveraging multi-view images generated by 2D diffusion models to extract 3D content.<n>However, 2D diffusion models often struggle to produce dense images with strong multi-view consistency.<n>We present CDI3D, a feed-forward framework designed for efficient, high-quality image-to-3D generation with view.
  arXiv  Detail & Related papers  (2025-03-11T03:08:43Z)
• F3D-Gaus: Feed-forward 3D-aware Generation on ImageNet with Cycle-Aggregative Gaussian Splatting [35.625593119642424]
  This paper tackles the problem of generalizable 3D-aware generation from monocular datasets.<n>We propose a novel feed-forward pipeline based on pixel-aligned Gaussian Splatting.<n>We also introduce a self-supervised cycle-aggregative constraint to enforce cross-view consistency in the learned 3D representation.
  arXiv  Detail & Related papers  (2025-01-12T04:44:44Z)
• GeoLRM: Geometry-Aware Large Reconstruction Model for High-Quality 3D Gaussian Generation [65.33726478659304]
  We introduce the Geometry-Aware Large Reconstruction Model (GeoLRM), an approach which can predict high-quality assets with 512k Gaussians and 21 input images in only 11 GB GPU memory. Previous works neglect the inherent sparsity of 3D structure and do not utilize explicit geometric relationships between 3D and 2D images. GeoLRM tackles these issues by incorporating a novel 3D-aware transformer structure that directly processes 3D points and uses deformable cross-attention mechanisms.
  arXiv  Detail & Related papers  (2024-06-21T17:49:31Z)
• ComboVerse: Compositional 3D Assets Creation Using Spatially-Aware Diffusion Guidance [76.7746870349809]
  We present ComboVerse, a 3D generation framework that produces high-quality 3D assets with complex compositions by learning to combine multiple models. Our proposed framework emphasizes spatial alignment of objects, compared with standard score distillation sampling.
  arXiv  Detail & Related papers  (2024-03-19T03:39:43Z)
• PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigm [111.16358607889609]
  We introduce a novel universal 3D pre-training framework designed to facilitate the acquisition of efficient 3D representation.<n>For the first time, PonderV2 achieves state-of-the-art performance on 11 indoor and outdoor benchmarks, implying its effectiveness.
  arXiv  Detail & Related papers  (2023-10-12T17:59:57Z)
• AutoDecoding Latent 3D Diffusion Models [95.7279510847827]
  We present a novel approach to the generation of static and articulated 3D assets that has a 3D autodecoder at its core. The 3D autodecoder framework embeds properties learned from the target dataset in the latent space. We then identify the appropriate intermediate volumetric latent space, and introduce robust normalization and de-normalization operations.
  arXiv  Detail & Related papers  (2023-07-07T17:59:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.