3D Shape Perception Integrates Intuitive Physics and Analysis-by-Synthesis

• URL: http://arxiv.org/abs/2301.03711v1
• Date: Mon, 9 Jan 2023 23:11:41 GMT
• Title: 3D Shape Perception Integrates Intuitive Physics and Analysis-by-Synthesis
• Authors: Ilker Yildirim, Max H. Siegel, Amir A. Soltani, Shraman Ray Chaudhari, Joshua B. Tenenbaum
• Abstract summary: We propose a framework for 3D shape perception that explains perception in both typical and atypical cases. Our results suggest that bottom-up deep neural network models are not fully adequate accounts of human shape perception.
• Score: 39.933479524063976
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Many surface cues support three-dimensional shape perception, but people can sometimes still see shape when these features are missing -- in extreme cases, even when an object is completely occluded, as when covered with a draped cloth. We propose a framework for 3D shape perception that explains perception in both typical and atypical cases as analysis-by-synthesis, or inference in a generative model of image formation: the model integrates intuitive physics to explain how shape can be inferred from deformations it causes to other objects, as in cloth-draping. Behavioral and computational studies comparing this account with several alternatives show that it best matches human observers in both accuracy and response times, and is the only model that correlates significantly with human performance on difficult discriminations. Our results suggest that bottom-up deep neural network models are not fully adequate accounts of human shape perception, and point to how machine vision systems might achieve more human-like robustness.

Related papers

• Learning Motion-Dependent Appearance for High-Fidelity Rendering of Dynamic Humans from a Single Camera [49.357174195542854]
  A key challenge of learning the dynamics of the appearance lies in the requirement of a prohibitively large amount of observations. We show that our method can generate a temporally coherent video of dynamic humans for unseen body poses and novel views given a single view video.
  arXiv  Detail & Related papers  (2022-03-24T00:22:03Z)
• LatentHuman: Shape-and-Pose Disentangled Latent Representation for Human Bodies [78.17425779503047]
  We propose a novel neural implicit representation for the human body. It is fully differentiable and optimizable with disentangled shape and pose latent spaces. Our model can be trained and fine-tuned directly on non-watertight raw data with well-designed losses.
  arXiv  Detail & Related papers  (2021-11-30T04:10:57Z)
• Identity-Disentangled Neural Deformation Model for Dynamic Meshes [8.826835863410109]
  We learn a neural deformation model that disentangles identity-induced shape variations from pose-dependent deformations using implicit neural functions. We propose two methods to integrate global pose alignment with our neural deformation model. Our method also outperforms traditional skeleton-driven models in reconstructing surface details such as palm prints or tendons without limitations from a fixed template.
  arXiv  Detail & Related papers  (2021-09-30T17:43:06Z)
• Detailed Avatar Recovery from Single Image [50.82102098057822]
  This paper presents a novel framework to recover emphdetailed avatar from a single image. We use the deep neural networks to refine the 3D shape in a Hierarchical Mesh Deformation framework. Our method can restore detailed human body shapes with complete textures beyond skinned models.
  arXiv  Detail & Related papers  (2021-08-06T03:51:26Z)
• Synthetic Training for Accurate 3D Human Pose and Shape Estimation in the Wild [27.14060158187953]
  This paper addresses the problem of monocular 3D human shape and pose estimation from an RGB image. We propose STRAPS, a system that uses proxy representations, such as silhouettes and 2D joints, as inputs to a shape and pose regression neural network. We show that STRAPS outperforms other state-of-the-art methods on SSP-3D in terms of shape prediction accuracy.
  arXiv  Detail & Related papers  (2020-09-21T16:39:04Z)
• Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction [123.62341095156611]
  Implicit functions represented as deep learning approximations are powerful for reconstructing 3D surfaces. Such features are essential in building flexible models for both computer graphics and computer vision. We present methodology that combines detail-rich implicit functions and parametric representations.
  arXiv  Detail & Related papers  (2020-07-22T13:46:14Z)
• Visual Grounding of Learned Physical Models [66.04898704928517]
  Humans intuitively recognize objects' physical properties and predict their motion, even when the objects are engaged in complicated interactions. We present a neural model that simultaneously reasons about physics and makes future predictions based on visual and dynamics priors. Experiments show that our model can infer the physical properties within a few observations, which allows the model to quickly adapt to unseen scenarios and make accurate predictions into the future.
  arXiv  Detail & Related papers  (2020-04-28T17:06:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.