IKOL: Inverse kinematics optimization layer for 3D human pose and shape estimation via Gauss-Newton differentiation

• URL: http://arxiv.org/abs/2302.01058v1
• Date: Thu, 2 Feb 2023 12:43:29 GMT
• Title: IKOL: Inverse kinematics optimization layer for 3D human pose and shape estimation via Gauss-Newton differentiation
• Authors: Juze Zhang, Ye Shi, Ye Shi, Lan Xu, Jingyi Yu, Jingya Wang
• Abstract summary: This paper presents an inverse kinematic layer (IKOL) for 3D human pose shape estimation. IKOL has a much over over than most existing regression-based methods. It provides a more accurate range of 3D human pose estimation.
• Score: 44.00115413716392
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents an inverse kinematic optimization layer (IKOL) for 3D human pose and shape estimation that leverages the strength of both optimization- and regression-based methods within an end-to-end framework. IKOL involves a nonconvex optimization that establishes an implicit mapping from an image's 3D keypoints and body shapes to the relative body-part rotations. The 3D keypoints and the body shapes are the inputs and the relative body-part rotations are the solutions. However, this procedure is implicit and hard to make differentiable. So, to overcome this issue, we designed a Gauss-Newton differentiation (GN-Diff) procedure to differentiate IKOL. GN-Diff iteratively linearizes the nonconvex objective function to obtain Gauss-Newton directions with closed form solutions. Then, an automatic differentiation procedure is directly applied to generate a Jacobian matrix for end-to-end training. Notably, the GN-Diff procedure works fast because it does not rely on a time-consuming implicit differentiation procedure. The twist rotation and shape parameters are learned from the neural networks and, as a result, IKOL has a much lower computational overhead than most existing optimization-based methods. Additionally, compared to existing regression-based methods, IKOL provides a more accurate mesh-image correspondence. This is because it iteratively reduces the distance between the keypoints and also enhances the reliability of the pose structures. Extensive experiments demonstrate the superiority of our proposed framework over a wide range of 3D human pose and shape estimation methods.

Related papers

• 3D Equivariant Pose Regression via Direct Wigner-D Harmonics Prediction [50.07071392673984]
  Existing methods learn 3D rotations parametrized in the spatial domain using angles or quaternions. We propose a frequency-domain approach that directly predicts Wigner-D coefficients for 3D rotation regression. Our method achieves state-of-the-art results on benchmarks such as ModelNet10-SO(3) and PASCAL3D+.
  arXiv  Detail & Related papers  (2024-11-01T12:50:38Z)
• Fast-SNARF: A Fast Deformer for Articulated Neural Fields [92.68788512596254]
  We propose a new articulation module for neural fields, Fast-SNARF, which finds accurate correspondences between canonical space and posed space. Fast-SNARF is a drop-in replacement in to our previous work, SNARF, while significantly improving its computational efficiency. Because learning of deformation maps is a crucial component in many 3D human avatar methods, we believe that this work represents a significant step towards the practical creation of 3D virtual humans.
  arXiv  Detail & Related papers  (2022-11-28T17:55:34Z)
• PLIKS: A Pseudo-Linear Inverse Kinematic Solver for 3D Human Body Estimation [10.50175010474078]
  We introduce PLIKS for reconstruction of a 3D mesh of the human body from a single 2D image. PLIKS is built on a linearized formulation of the parametric SMPL model. We present evaluations which confirm that PLIKS achieves more accurate reconstruction with greater than 10% improvement.
  arXiv  Detail & Related papers  (2022-11-21T18:54:12Z)
• Learned Vertex Descent: A New Direction for 3D Human Model Fitting [64.04726230507258]
  We propose a novel optimization-based paradigm for 3D human model fitting on images and scans. Our approach is able to capture the underlying body of clothed people with very different body shapes, achieving a significant improvement compared to state-of-the-art. LVD is also applicable to 3D model fitting of humans and hands, for which we show a significant improvement to the SOTA with a much simpler and faster method.
  arXiv  Detail & Related papers  (2022-05-12T17:55:51Z)
• Deep convolutional neural network for shape optimization using level-set approach [0.0]
  This article presents a reduced-order modeling methodology for shape optimization applications via deep convolutional neural networks (CNNs) A CNN-based reduced-order model (ROM) is constructed in a completely data-driven manner, and suited for non-intrusive applications.
  arXiv  Detail & Related papers  (2022-01-17T04:41:51Z)
• Equivalent Distance Geometry Error for Molecular Conformation Comparison [26.331322944298208]
  We propose Equivalent Distance Geometry Error (EDGE) to calculate the differential discrepancy between conformations. In the improved version of our method, the optimization features minimizing linear transformations of atom-pair distances within 3-hop.
  arXiv  Detail & Related papers  (2021-11-13T09:04:55Z)
• Differentiable Rendering with Perturbed Optimizers [85.66675707599782]
  Reasoning about 3D scenes from their 2D image projections is one of the core problems in computer vision. Our work highlights the link between some well-known differentiable formulations and randomly smoothed renderings. We apply our method to 3D scene reconstruction and demonstrate its advantages on the tasks of 6D pose estimation and 3D mesh reconstruction.
  arXiv  Detail & Related papers  (2021-10-18T08:56:23Z)
• Robust Regression via Model Based Methods [13.300549123177705]
  We propose an algorithm inspired by so-called model-based optimization (MBO) [35, 36], which replaces a non-objective with a convex model function. We apply this to robust regression, proposing SADM, a function of the Online Alternating Direction Method of Multipliers (OOADM) [50] to solve the inner optimization in MBO. Finally, we demonstrate experimentally (a) the robustness of l_p norms to outliers and (b) the efficiency of our proposed model-based algorithms in comparison with methods on autoencoders and multi-target regression.
  arXiv  Detail & Related papers  (2021-06-20T21:45:35Z)
• Revitalizing Optimization for 3D Human Pose and Shape Estimation: A Sparse Constrained Formulation [21.710205047008916]
  We propose a novel sparse constrained formulation and from it derive a real-time optimization method for 3D human pose and shape estimation. We show that this computation scales linearly with the number of joints of a complex 3D human model, in contrast to prior work where it scales cubically due to their dense unconstrained formulation. We present a real-time motion capture framework that estimates 3D human poses and shapes from a single image at over 30 FPS.
  arXiv  Detail & Related papers  (2021-05-28T16:44:56Z)
• Cogradient Descent for Bilinear Optimization [124.45816011848096]
  We introduce a Cogradient Descent algorithm (CoGD) to address the bilinear problem. We solve one variable by considering its coupling relationship with the other, leading to a synchronous gradient descent. Our algorithm is applied to solve problems with one variable under the sparsity constraint.
  arXiv  Detail & Related papers  (2020-06-16T13:41:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.