FactorizedHMR: A Hybrid Framework for Video Human Mesh Recovery

• URL: http://arxiv.org/abs/2605.14854v2
• Date: Sat, 16 May 2026 00:13:57 GMT
• Title: FactorizedHMR: A Hybrid Framework for Video Human Mesh Recovery
• Abstract summary: Human Mesh Recovery (HMR) is fundamentally ambiguous.<n>We propose FactorizedHMR, a two-stage framework that treats these two regimes differently.<n>We show that FactorizedHMR remains competitive with strong baselines across camera-space and world-space benchmarks.
• Score: 6.1625435030904745
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Human Mesh Recovery (HMR) is fundamentally ambiguous: under occlusion or weak depth cues, multiple 3D bodies can explain the same image evidence. This ambiguity is not uniform across the body, as torso pose and root structure are often relatively well constrained, whereas distal articulations such as the arms and legs are more uncertain. Building on this observation, we propose FactorizedHMR, a two-stage framework that treats these two regimes differently. A deterministic regression module first recovers a stable torso-root anchor, and a probabilistic flow-matching module then completes the remaining non-torso articulation. To make this completion reliable, we combine a composite target representation with geometry-aware supervision and feature-aware classifier-free guidance, preserving the torso-root anchor while improving single-reference recovery of ambiguity-prone articulation. We also introduce a synthetic data pipeline that provides the paired image-camera-motion supervision under diverse viewpoints. Across camera-space and world-space benchmarks, FactorizedHMR remains competitive with strong baselines, with the clearest gains in occlusion-heavy recovery and drift-sensitive world-space metrics.

Related papers

• High-Dimensional Latents Should Be Diagnosed Through Phase Structure [56.362776482614976]
  We study autoencoder and variational-autoencoder latent spaces through the lens of spin-glass theory.<n>We formalize a latent-space spin-glass dictionary for a fixed decoder.<n>We show that driving the latent system toward the edge-of-stability of the topological trivialization regime has concrete downstream consequences.
  arXiv  Detail & Related papers  (2026-05-23T06:17:23Z)
• ResiHMR: Residual-Limb Aware Single-Image 3D Human Mesh Recovery for Individuals with Limb Loss [28.590347498910006]
  We present ResiHMR, a residual-limb aware framework for single-image 3D human modeling.<n>To the best of our knowledge, this is the first single-image HMR system that explicitly reconstructs residual-limb surfaces.
  arXiv  Detail & Related papers  (2026-04-30T15:42:17Z)
• Discriminative-Generative Synergy for Occlusion Robust 3D Human Mesh Recovery [5.966636064805421]
  3D human mesh recovery from monocular RGB images aims to estimate anatomically plausible 3D human models for downstream applications.<n>We propose a brain-inspired framework that integrates the discriminative power of vision transformers with the generative capability of conditional diffusion models.
  arXiv  Detail & Related papers  (2026-04-20T20:31:54Z)
• VLM-Guided Group Preference Alignment for Diffusion-based Human Mesh Recovery [75.62565146049015]
  We introduce a dual-memory augmented HMR critique agent with self-reflection to produce context-aware quality scores for predicted meshes.<n>These scores distill fine-grained cues about 3D human motion structure, physical feasibility, and alignment with the input image.<n>We propose a group preference alignment framework for finetuning diffusion-based HMR models.
  arXiv  Detail & Related papers  (2026-02-22T13:19:06Z)
• OMG-Agent: Toward Robust Missing Modality Generation with Decoupled Coarse-to-Fine Agentic Workflows [9.617220633655716]
  We present textbfunderlineOmni-textbfunderlineModality textbfunderlineGeneration Agent (textbfOMG-Agent)
  arXiv  Detail & Related papers  (2026-02-04T02:25:40Z)
• Towards Geometry-Aware and Motion-Guided Video Human Mesh Recovery [60.51998732898099]
  We introduce HMRMamba, a new paradigm for 3D Human Mesh Recovery.<n>It pioneers the use of Structured State Space Models for their efficiency and long-range modeling prowess.<n>Our framework is distinguished by two core contributions. First, the Geometry-Aware Lifting Module, featuring a novel dual-scan Mamba architecture.
  arXiv  Detail & Related papers  (2026-01-29T08:05:02Z)
• Bridging Information Asymmetry: A Hierarchical Framework for Deterministic Blind Face Restoration [31.878334664450776]
  We present textbfPrefRestore, a hierarchical framework that integrates discrete semantic logic with continuous texture generation.<n>Our methodology fundamentally addresses this information disparity through two complementary strategies.<n>Pref-Restore achieves state-of-the-art performance across synthetic and real-world benchmarks.
  arXiv  Detail & Related papers  (2026-01-27T11:50:31Z)
• BHaRNet: Reliability-Aware Body-Hand Modality Expertized Networks for Fine-grained Skeleton Action Recognition [12.973657570368317]
  This work presents a probabilistic dual-stream framework that unifies reliability modeling and multi-modal integration, generalizing expertized learning under uncertainty across both intra-skeleton and cross-modal domains.<n>The framework comprises three key components: (1) a calibration-free preprocessing pipeline that removes canonical-space transformations and learns directly from native coordinates; (2) a probabilistic Noisy-OR fusion that stabilizes reliability-aware dual-stream learning without requiring explicit confidence supervision; and (3) an intra- to cross-modal ensemble that couples four skeleton modalities (Joint, Bone, Joint Motion, and Bone Motion) to RGB representations, bridging structural and visual motion
  arXiv  Detail & Related papers  (2026-01-01T15:13:11Z)
• Robust-R1: Degradation-Aware Reasoning for Robust Visual Understanding [54.05243949024302]
  Existing robust MLLMs rely on implicit training/adaptation that focuses solely on visual encoder generalization.<n>We propose Robust-R1, a novel framework that explicitly models visual degradations through structured reasoning chains.<n>Our approach integrates: (i) supervised fine-tuning for degradation-aware reasoning foundations, (ii) reward-driven alignment for accurately perceiving degradation parameters, and (iii) dynamic reasoning depth scaling adapted to degradation intensity.
  arXiv  Detail & Related papers  (2025-12-19T12:56:17Z)
• Learning to Align and Refine: A Foundation-to-Diffusion Framework for Occlusion-Robust Two-Hand Reconstruction [50.952228546326516]
  Two-hand reconstruction from monocular images faces persistent challenges due to complex and dynamic hand postures.<n>Existing approaches struggle with such alignment issues, often resulting in misalignment and penetration artifacts.<n>We propose a dual-stage Foundation-to-Diffusion framework that precisely align 2D prior guidance from vision foundation models.
  arXiv  Detail & Related papers  (2025-03-22T14:42:27Z)
• MotioNet: 3D Human Motion Reconstruction from Monocular Video with Skeleton Consistency [72.82534577726334]
  We introduce MotioNet, a deep neural network that directly reconstructs the motion of a 3D human skeleton from monocular video. Our method is the first data-driven approach that directly outputs a kinematic skeleton, which is a complete, commonly used, motion representation.
  arXiv  Detail & Related papers  (2020-06-22T08:50:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.