Abstract: This work addresses hand mesh recovery from a single RGB image. In contrast
to most of the existing approaches where the parametric hand models are
employed as the prior, we show that the hand mesh can be learned directly from
the input image. We propose a new type of GAN called Im2Mesh GAN to learn the
mesh through end-to-end adversarial training. By interpreting the mesh as a
graph, our model is able to capture the topological relationship among the mesh
vertices. We also introduce a 3D surface descriptor into the GAN architecture
to further capture the 3D features associated. We experiment two approaches
where one can reap the benefits of coupled groundtruth data availability of
images and the corresponding meshes, while the other combats the more
challenging problem of mesh estimations without the corresponding groundtruth.
Through extensive evaluations we demonstrate that the proposed method
outperforms the state-of-the-art.