Detailed Geometry and Appearance from Opportunistic Motion

• URL: http://arxiv.org/abs/2603.26665v1
• Date: Fri, 27 Mar 2026 17:59:16 GMT
• Title: Detailed Geometry and Appearance from Opportunistic Motion
• Abstract summary: Reconstructing 3D geometry and appearance from a sparse set of fixed cameras is a foundational task with broad applications.<n>We show that the static cameras effectively orbit'' the object in its local coordinate frame, providing additional virtual viewpoints.<n> Harnessing this object motion, however, poses two challenges: the tight coupling of object pose and geometry estimation and the complex appearance variations of a moving object under static illumination.
• Score: 44.38825288921379
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reconstructing 3D geometry and appearance from a sparse set of fixed cameras is a foundational task with broad applications, yet it remains fundamentally constrained by the limited viewpoints. We show that this bound can be broken by exploiting opportunistic object motion: as a person manipulates an object~(e.g., moving a chair or lifting a mug), the static cameras effectively ``orbit'' the object in its local coordinate frame, providing additional virtual viewpoints. Harnessing this object motion, however, poses two challenges: the tight coupling of object pose and geometry estimation and the complex appearance variations of a moving object under static illumination. We address these by formulating a joint pose and shape optimization using 2D Gaussian splatting with alternating minimization of 6DoF trajectories and primitive parameters, and by introducing a novel appearance model that factorizes diffuse and specular components with reflected directional probing within the spherical harmonics space. Extensive experiments on synthetic and real-world datasets with extremely sparse viewpoints demonstrate that our method recovers significantly more accurate geometry and appearance than state-of-the-art baselines.

Related papers

• SpatialDiff: 3D-Aware Object Movement via Implicit Spatial Modeling [60.4038189368113]
  We introduce Implicit 3D Spatial Modeling, which introduces 3D prior knowledge and enables the model to internally build a comprehensive understanding of the three-dimensional spatial structure.<n>We also present Global Spatial Supervision, which constrains the latent spatial features to enable the model to perceive changes in object spatial positions caused by editing operations.<n> Experimental results demonstrate that our method significantly improves the accuracy and fidelity of spatial movement in complex scenes.
  arXiv  Detail & Related papers  (2026-08-22T04:46:08Z)
• RoPEMover: Depth-Aware Object Relocation via Positional Embeddings [25.069960502722296]
  We introduce a geometry-aware object motion method that operates directly on the positional representations of diffusion transformers.<n>We extend 2D RoPE into a depth-aware formulation that encodes 3D spatial structure.<n>Our model is trained using synthetic data combined with a small set of real images via parameter-efficient fine-tuning.
  arXiv  Detail & Related papers  (2026-06-25T17:45:20Z)
• Towards Realistic and Consistent Orbital Video Generation via 3D Foundation Priors [61.34273238077091]
  We present a novel method for generating orbital videos from a single image of an object.<n>Our method achieves superior visual quality, shape realism and multi-view consistency compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-04-14T05:35:46Z)
• Articulat3D: Reconstructing Articulated Digital Twins From Monocular Videos with Geometric and Motion Constraints [21.83046776294786]
  Articulat3D is a novel framework that constructs digital twins from casually captured monocular videos.<n>We first propose Motion Prior-Driven Initialization, which leverages 3D point tracks to exploit the low-dimensional structure of articulated motion.<n>We then introduce Geometric and Motion Constraints Refinement, which enforces physically plausible articulation.<n>Experiments demonstrate that Articulat3D achieves state-of-the-art performance on synthetic benchmarks and real-world casually captured monocular videos.
  arXiv  Detail & Related papers  (2026-03-12T06:59:44Z)
• PAOLI: Pose-free Articulated Object Learning from Sparse-view Images [27.16160315662701]
  We present a novel framework for learning articulated object representations from sparse-view, unposed images.<n>Our approach operates with as few as four views per articulation and no camera supervision.
  arXiv  Detail & Related papers  (2025-09-04T14:51:03Z)
• GaussianArt: Unified Modeling of Geometry and Motion for Articulated Objects [4.717906057951389]
  We introduce a unified representation that jointly models geometry and motion using articulated 3D Gaussians.<n>This formulation improves robustness in motion decomposition and supports articulated objects with up to 20 parts.<n>We show that our method consistently achieves superior accuracy in part-level geometry reconstruction and motion estimation across a broad range of object types.
  arXiv  Detail & Related papers  (2025-08-20T17:59:08Z)
• Part$^{2}$GS: Part-aware Modeling of Articulated Objects using 3D Gaussian Splatting [9.432567695900184]
  Part$2$GS is a novel framework for modeling articulated digital twins of multi-part objects with high-fidelity geometry.<n>To ensure physically consistent motion, we propose a motion-aware canonical representation guided by physics-based constraints.<n>We show that Part$2$GS consistently outperforms state-of-the-art methods by up to 10$times$ in Chamfer Distance for movable parts.
  arXiv  Detail & Related papers  (2025-06-20T17:59:12Z)
• Dynamic Scene Understanding through Object-Centric Voxelization and Neural Rendering [57.895846642868904]
  We present a 3D generative model named DynaVol-S for dynamic scenes that enables object-centric learning.<n>voxelization infers per-object occupancy probabilities at individual spatial locations.<n>Our approach integrates 2D semantic features to create 3D semantic grids, representing the scene through multiple disentangled voxel grids.
  arXiv  Detail & Related papers  (2024-07-30T15:33:58Z)
• Decaf: Monocular Deformation Capture for Face and Hand Interactions [77.75726740605748]
  This paper introduces the first method that allows tracking human hands interacting with human faces in 3D from single monocular RGB videos. We model hands as articulated objects inducing non-rigid face deformations during an active interaction. Our method relies on a new hand-face motion and interaction capture dataset with realistic face deformations acquired with a markerless multi-view camera system.
  arXiv  Detail & Related papers  (2023-09-28T17:59:51Z)
• Towards Scalable Multi-View Reconstruction of Geometry and Materials [27.660389147094715]
  We propose a novel method for joint recovery of camera pose, object geometry and spatially-varying Bidirectional Reflectance Distribution Function (svBRDF) of 3D scenes. The input are high-resolution RGBD images captured by a mobile, hand-held capture system with point lights for active illumination.
  arXiv  Detail & Related papers  (2023-06-06T15:07:39Z)
• Object Wake-up: 3-D Object Reconstruction, Animation, and in-situ Rendering from a Single Image [58.69732754597448]
  Given a picture of a chair, could we extract the 3-D shape of the chair, animate its plausible articulations and motions, and render in-situ in its original image space? We devise an automated approach to extract and manipulate articulated objects in single images.
  arXiv  Detail & Related papers  (2021-08-05T16:20:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.