Discriminative-Generative Synergy for Occlusion Robust 3D Human Mesh Recovery

• URL: http://arxiv.org/abs/2604.21712v1
• Date: Mon, 20 Apr 2026 20:31:54 GMT
• Title: Discriminative-Generative Synergy for Occlusion Robust 3D Human Mesh Recovery
• Abstract summary: 3D human mesh recovery from monocular RGB images aims to estimate anatomically plausible 3D human models for downstream applications.<n>We propose a brain-inspired framework that integrates the discriminative power of vision transformers with the generative capability of conditional diffusion models.
• Score: 5.966636064805421
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: 3D human mesh recovery from monocular RGB images aims to estimate anatomically plausible 3D human models for downstream applications, but remains challenging under partial or severe occlusions. Regression-based methods are efficient yet often produce implausible or inaccurate results in unconstrained scenarios, while diffusion-based methods provide strong generative priors for occluded regions but may weaken fidelity to rare poses due to over-reliance on generation. To address these limitations, we propose a brain-inspired synergistic framework that integrates the discriminative power of vision transformers with the generative capability of conditional diffusion models. Specifically, the ViT-based pathway extracts deterministic visual cues from visible regions, while the diffusion-based pathway synthesizes structurally coherent human body representations. To effectively bridge the two pathways, we design a diverse-consistent feature learning module to align discriminative features with generative priors, and a cross-attention multi-level fusion mechanism to enable bidirectional interaction across semantic levels. Experiments on standard benchmarks demonstrate that our method achieves superior performance on key metrics and shows strong robustness in complex real-world scenarios.

Related papers

• DisPOSE: Projected Polystochastic Diffusion for Self-Supervised Multi-View 3D Human Pose Estimation [58.47973015036709]
  DisPOSE is a self-supervised framework that approximates the inherently discrete multi-view person-assignment problem.<n>By employing differentiable Sinkhorn projections during denoising, our model learns to guide solutions toward valid and feasible assignments.<n>The proposed approach outperforms current state-of-the-art self-supervised methods on standard datasets.
  arXiv  Detail & Related papers  (2026-06-05T16:14:39Z)
• DUET: Dual-Paradigm Adaptive Expert Triage with Single-cell Inductive Prior for Spatial Transcriptomics Prediction [6.028508407069093]
  Inferring spatially resolved gene expression from histology images offers a cost-effective complement to spatial transcriptomics.<n>We propose DUET, a novel dual-paradigm framework that synergizes parametric prediction and memory-based retrieval under cellular inductive priors.
  arXiv  Detail & Related papers  (2026-05-13T20:43:29Z)
• VLM-Guided Group Preference Alignment for Diffusion-based Human Mesh Recovery [75.62565146049015]
  We introduce a dual-memory augmented HMR critique agent with self-reflection to produce context-aware quality scores for predicted meshes.<n>These scores distill fine-grained cues about 3D human motion structure, physical feasibility, and alignment with the input image.<n>We propose a group preference alignment framework for finetuning diffusion-based HMR models.
  arXiv  Detail & Related papers  (2026-02-22T13:19:06Z)
• DNA: Uncovering Universal Latent Forgery Knowledge [39.19379714306656]
  forgery detection capability is already encoded within pre-trained models.<n>DNA framework employs a coarse-to-fine excavation mechanism.<n>Hifi-Gen is a high-fidelity synthetic benchmark built upon the very latest models.
  arXiv  Detail & Related papers  (2026-01-30T03:48:30Z)
• Dual Conditioned Motion Diffusion for Pose-Based Video Anomaly Detection [12.100563798908777]
  Video Anomaly Detection (VAD) is essential for computer vision research.<n>Existing VAD methods utilize either reconstruction-based or prediction-based frameworks.<n>We address pose-based video anomaly detection and introduce a novel framework called Dual Conditioned Motion Diffusion.
  arXiv  Detail & Related papers  (2024-12-23T01:31:39Z)
• DPMesh: Exploiting Diffusion Prior for Occluded Human Mesh Recovery [71.6345505427213]
  DPMesh is an innovative framework for occluded human mesh recovery. It capitalizes on the profound diffusion prior about object structure and spatial relationships embedded in a pre-trained text-to-image diffusion model.
  arXiv  Detail & Related papers  (2024-04-01T18:59:13Z)
• D-SCo: Dual-Stream Conditional Diffusion for Monocular Hand-Held Object Reconstruction [74.49121940466675]
  We introduce centroid-fixed dual-stream conditional diffusion for monocular hand-held object reconstruction. First, to avoid the object centroid from deviating, we utilize a novel hand-constrained centroid fixing paradigm. Second, we introduce a dual-stream denoiser to semantically and geometrically model hand-object interactions.
  arXiv  Detail & Related papers  (2023-11-23T20:14:50Z)
• Reference-Free Isotropic 3D EM Reconstruction using Diffusion Models [8.590026259176806]
  We propose a diffusion-model-based framework that overcomes the limitations of requiring reference data or prior knowledge about the degradation process. Our approach utilizes 2D diffusion models to consistently reconstruct 3D volumes and is well-suited for highly downsampled data.
  arXiv  Detail & Related papers  (2023-08-03T07:57:02Z)
• Progressive Multi-view Human Mesh Recovery with Self-Supervision [68.60019434498703]
  Existing solutions typically suffer from poor generalization performance to new settings. We propose a novel simulation-based training pipeline for multi-view human mesh recovery.
  arXiv  Detail & Related papers  (2022-12-10T06:28:29Z)
• Uncertainty-Aware Adaptation for Self-Supervised 3D Human Pose Estimation [70.32536356351706]
  We introduce MRP-Net that constitutes a common deep network backbone with two output heads subscribing to two diverse configurations. We derive suitable measures to quantify prediction uncertainty at both pose and joint level. We present a comprehensive evaluation of the proposed approach and demonstrate state-of-the-art performance on benchmark datasets.
  arXiv  Detail & Related papers  (2022-03-29T07:14:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.