Symmetry Strikes Back: From Single-Image Symmetry Detection to 3D Generation

• URL: http://arxiv.org/abs/2411.17763v1
• Date: Tue, 26 Nov 2024 04:14:31 GMT
• Title: Symmetry Strikes Back: From Single-Image Symmetry Detection to 3D Generation
• Authors: Xiang Li, Zixuan Huang, Anh Thai, James M. Rehg,
• Abstract summary: We introduce Reflect3D, a scalable, zero-shot symmetry detector capable of robust generalization to diverse and real-world scenarios. We show the practical benefit of incorporating detected symmetry into single-image 3D generation pipelines.
• Score: 29.732780338284353
• License:
• Abstract: Symmetry is a ubiquitous and fundamental property in the visual world, serving as a critical cue for perception and structure interpretation. This paper investigates the detection of 3D reflection symmetry from a single RGB image, and reveals its significant benefit on single-image 3D generation. We introduce Reflect3D, a scalable, zero-shot symmetry detector capable of robust generalization to diverse and real-world scenarios. Inspired by the success of foundation models, our method scales up symmetry detection with a transformer-based architecture. We also leverage generative priors from multi-view diffusion models to address the inherent ambiguity in single-view symmetry detection. Extensive evaluations on various data sources demonstrate that Reflect3D establishes a new state-of-the-art in single-image symmetry detection. Furthermore, we show the practical benefit of incorporating detected symmetry into single-image 3D generation pipelines through a symmetry-aware optimization process. The integration of symmetry significantly enhances the structural accuracy, cohesiveness, and visual fidelity of the reconstructed 3D geometry and textures, advancing the capabilities of 3D content creation.

Related papers

• PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting [54.7468067660037]
  PF3plat sets a new state-of-the-art across all benchmarks, supported by comprehensive ablation studies validating our design choices. Our framework capitalizes on fast speed, scalability, and high-quality 3D reconstruction and view synthesis capabilities of 3DGS.
  arXiv  Detail & Related papers  (2024-10-29T15:28:15Z)
• GeoGS3D: Single-view 3D Reconstruction via Geometric-aware Diffusion Model and Gaussian Splatting [81.03553265684184]
  We introduce GeoGS3D, a framework for reconstructing detailed 3D objects from single-view images. We propose a novel metric, Gaussian Divergence Significance (GDS), to prune unnecessary operations during optimization. Experiments demonstrate that GeoGS3D generates images with high consistency across views and reconstructs high-quality 3D objects.
  arXiv  Detail & Related papers  (2024-03-15T12:24:36Z)
• Partial Symmetry Detection for 3D Geometry using Contrastive Learning with Geodesic Point Cloud Patches [10.48309709793733]
  We propose to learn rotation, reflection, translation and scale invariant local shape features for geodesic point cloud patches. We show that our approach is able to extract multiple valid solutions for this ambiguous problem. We incorporate the detected symmetries together with a region growing algorithm to demonstrate a downstream task.
  arXiv  Detail & Related papers  (2023-12-13T15:48:50Z)
• Beyond 3DMM: Learning to Capture High-fidelity 3D Face Shape [77.95154911528365]
  3D Morphable Model (3DMM) fitting has widely benefited face analysis due to its strong 3D priori. Previous reconstructed 3D faces suffer from degraded visual verisimilitude due to the loss of fine-grained geometry. This paper proposes a complete solution to capture the personalized shape so that the reconstructed shape looks identical to the corresponding person.
  arXiv  Detail & Related papers  (2022-04-09T03:46:18Z)
• A Model for Multi-View Residual Covariances based on Perspective Deformation [88.21738020902411]
  We derive a model for the covariance of the visual residuals in multi-view SfM, odometry and SLAM setups. We validate our model with synthetic and real data and integrate it into photometric and feature-based Bundle Adjustment.
  arXiv  Detail & Related papers  (2022-02-01T21:21:56Z)
• SIDER: Single-Image Neural Optimization for Facial Geometric Detail Recovery [54.64663713249079]
  SIDER is a novel photometric optimization method that recovers detailed facial geometry from a single image in an unsupervised manner. In contrast to prior work, SIDER does not rely on any dataset priors and does not require additional supervision from multiple views, lighting changes or ground truth 3D shape.
  arXiv  Detail & Related papers  (2021-08-11T22:34:53Z)
• Recurrently Estimating Reflective Symmetry Planes from Partial Pointclouds [5.098175145801009]
  We present an alternative novel encoding that instead slices the data along the height dimension and passes it sequentially to a 2D convolutional recurrent regression scheme. We show that our approach has an accuracy comparable to state-of-the-art techniques on the task of planar reflective symmetry estimation on full synthetic objects.
  arXiv  Detail & Related papers  (2021-06-30T15:26:15Z)
• NeRD: Neural 3D Reflection Symmetry Detector [27.626579746101292]
  We present NeRD, a Neural 3D Reflection Symmetry Detector. We first enumerate the symmetry planes with a coarse-to-fine strategy and then find the best ones by building 3D cost volumes. Our experiments show that the symmetry planes detected with our method are significantly more accurate than the planes from direct CNN regression.
  arXiv  Detail & Related papers  (2021-04-19T17:25:51Z)
• SymmetryNet: Learning to Predict Reflectional and Rotational Symmetries of 3D Shapes from Single-View RGB-D Images [26.38270361331076]
  We propose an end-to-end deep neural network which is able to predict both reflectional and rotational symmetries of 3D objects. We also contribute a benchmark of 3D symmetry detection based on single-view RGB-D images.
  arXiv  Detail & Related papers  (2020-08-02T14:10:09Z)
• PaMIR: Parametric Model-Conditioned Implicit Representation for Image-based Human Reconstruction [67.08350202974434]
  We propose Parametric Model-Conditioned Implicit Representation (PaMIR), which combines the parametric body model with the free-form deep implicit function. We show that our method achieves state-of-the-art performance for image-based 3D human reconstruction in the cases of challenging poses and clothing types.
  arXiv  Detail & Related papers  (2020-07-08T02:26:19Z)
• Learning to Detect 3D Reflection Symmetry for Single-View Reconstruction [32.14605731030579]
  3D reconstruction from a single RGB image is a challenging problem in computer vision. Previous methods are usually solely data-driven, which lead to inaccurate 3D shape recovery and limited generalization capability. We present a geometry-based end-to-end deep learning framework that first detects the mirror plane of reflection symmetry that commonly exists in man-made objects and then predicts depth maps by finding the intra-image pixel-wise correspondence of the symmetry.
  arXiv  Detail & Related papers  (2020-06-17T17:58:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.