Distribution-Aligned Diffusion for Human Mesh Recovery

• URL: http://arxiv.org/abs/2308.13369v3
• Date: Wed, 25 Oct 2023 03:32:42 GMT
• Title: Distribution-Aligned Diffusion for Human Mesh Recovery
• Authors: Lin Geng Foo, Jia Gong, Hossein Rahmani, Jun Liu
• Abstract summary: We propose a diffusion-based approach for human mesh recovery. We propose a Human Mesh Diffusion (HMDiff) framework which frames mesh recovery as a reverse diffusion process. Our method achieves state-of-the-art performance on three widely used datasets.
• Score: 16.64567393672489
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recovering a 3D human mesh from a single RGB image is a challenging task due to depth ambiguity and self-occlusion, resulting in a high degree of uncertainty. Meanwhile, diffusion models have recently seen much success in generating high-quality outputs by progressively denoising noisy inputs. Inspired by their capability, we explore a diffusion-based approach for human mesh recovery, and propose a Human Mesh Diffusion (HMDiff) framework which frames mesh recovery as a reverse diffusion process. We also propose a Distribution Alignment Technique (DAT) that infuses prior distribution information into the mesh distribution diffusion process, and provides useful prior knowledge to facilitate the mesh recovery task. Our method achieves state-of-the-art performance on three widely used datasets. Project page: https://gongjia0208.github.io/HMDiff/.

Related papers

• DPMesh: Exploiting Diffusion Prior for Occluded Human Mesh Recovery [71.6345505427213]
  DPMesh is an innovative framework for occluded human mesh recovery. It capitalizes on the profound diffusion prior about object structure and spatial relationships embedded in a pre-trained text-to-image diffusion model.
  arXiv  Detail & Related papers  (2024-04-01T18:59:13Z)
• Bayesian Diffusion Models for 3D Shape Reconstruction [54.69889488052155]
  We present a prediction algorithm that performs effective Bayesian inference by tightly coupling the top-down (prior) information with the bottom-up (data-driven) procedure. We show the effectiveness of BDM on the 3D shape reconstruction task.
  arXiv  Detail & Related papers  (2024-03-11T17:55:53Z)
• Learn to Optimize Denoising Scores for 3D Generation: A Unified and Improved Diffusion Prior on NeRF and 3D Gaussian Splatting [60.393072253444934]
  We propose a unified framework aimed at enhancing the diffusion priors for 3D generation tasks. We identify a divergence between the diffusion priors and the training procedures of diffusion models that substantially impairs the quality of 3D generation.
  arXiv  Detail & Related papers  (2023-12-08T03:55:34Z)
• Generative Approach for Probabilistic Human Mesh Recovery using Diffusion Models [33.2565018922113]
  This work focuses on the problem of reconstructing a 3D human body mesh from a given 2D image. We propose a generative approach framework, called "Diffusion-based Human Mesh Recovery (Diff-HMR)"
  arXiv  Detail & Related papers  (2023-08-05T22:23:04Z)
• Low-Light Image Enhancement with Wavelet-based Diffusion Models [50.632343822790006]
  Diffusion models have achieved promising results in image restoration tasks, yet suffer from time-consuming, excessive computational resource consumption, and unstable restoration. We propose a robust and efficient Diffusion-based Low-Light image enhancement approach, dubbed DiffLL.
  arXiv  Detail & Related papers  (2023-06-01T03:08:28Z)
• CamoDiffusion: Camouflaged Object Detection via Conditional Diffusion Models [72.93652777646233]
  Camouflaged Object Detection (COD) is a challenging task in computer vision due to the high similarity between camouflaged objects and their surroundings. We propose a new paradigm that treats COD as a conditional mask-generation task leveraging diffusion models. Our method, dubbed CamoDiffusion, employs the denoising process of diffusion models to iteratively reduce the noise of the mask.
  arXiv  Detail & Related papers  (2023-05-29T07:49:44Z)
• Deceptive-NeRF/3DGS: Diffusion-Generated Pseudo-Observations for High-Quality Sparse-View Reconstruction [60.52716381465063]
  We introduce Deceptive-NeRF/3DGS to enhance sparse-view reconstruction with only a limited set of input images. Specifically, we propose a deceptive diffusion model turning noisy images rendered from few-view reconstructions into high-quality pseudo-observations. Our system progressively incorporates diffusion-generated pseudo-observations into the training image sets, ultimately densifying the sparse input observations by 5 to 10 times.
  arXiv  Detail & Related papers  (2023-05-24T14:00:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.