Free3D: 3D Human Motion Emerges from Single-View 2D Supervision

• URL: http://arxiv.org/abs/2511.11368v1
• Date: Fri, 14 Nov 2025 14:49:19 GMT
• Title: Free3D: 3D Human Motion Emerges from Single-View 2D Supervision
• Abstract summary: Free3D is a framework that synthesizes realistic 3D motions without any 3D motion annotations.<n> trained entirely on 2D motion data, Free3D generates diverse, temporally coherent, and semantically aligned 3D motions.
• Score: 24.238045609839983
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Recent 3D human motion generation models demonstrate remarkable reconstruction accuracy yet struggle to generalize beyond training distributions. This limitation arises partly from the use of precise 3D supervision, which encourages models to fit fixed coordinate patterns instead of learning the essential 3D structure and motion semantic cues required for robust generalization.To overcome this limitation, we propose Free3D, a framework that synthesizes realistic 3D motions without any 3D motion annotations. Free3D introduces a Motion-Lifting Residual Quantized VAE (ML-RQ) that maps 2D motion sequences into 3D-consistent latent spaces, and a suite of 3D-free regularization objectives enforcing view consistency, orientation coherence, and physical plausibility. Trained entirely on 2D motion data, Free3D generates diverse, temporally coherent, and semantically aligned 3D motions, achieving performance comparable to or even surpassing fully 3D-supervised counterparts. These results suggest that relaxing explicit 3D supervision encourages stronger structural reasoning and generalization, offering a scalable and data-efficient paradigm for 3D motion generation.

Related papers

• SemanticSlider3D: Training-Free Continuous Semantic Editing for 3D Objects [15.020969387249872]
  Fine-grained control over continuous semantic attributes of 3D objects is essential for 3D content creation.<n>We present SemanticSlider3D, a technique for continuous semantic attribute editing of 3D objects.<n>Our pipeline constructs a semantic editing direction in the latent space of a state-of-the-art 3D generation model.
  arXiv  Detail & Related papers  (2026-08-19T05:35:38Z)
• Follow Your Track: Precise Skeleton Animation Controlled by 3D Trajectories [93.39170866007431]
  4D generation aims to animate 3D objects with realistic motion, holding great promise for applications.<n>Existing methods typically decouple 3D asset generation from motion synthesis.<n>We present textttACT, a trajectory-conditioned framework for topology-general skeletal animation.
  arXiv  Detail & Related papers  (2026-06-24T03:18:10Z)
• VideoMDM: Towards 3D Human Motion Generation From 2D Supervision [13.408796377211559]
  VideoMDM is a framework that trains 3D human motion priors directly from accurate 2D poses extracted from monocular videos.<n>A pretrained 2D-to-3D lifter provides approximate 3D pose sequences that serve as a noisy teacher.<n>We show that, under mild assumptions, a depth-weighted 2D reprojection loss is equivalent in expectation to direct 3D supervision.
  arXiv  Detail & Related papers  (2026-06-11T13:49:23Z)
• 3D-Aware Implicit Motion Control for View-Adaptive Human Video Generation [29.389246008057473]
  2D poses rigidly bind motion to the driving viewpoint, precluding novel-view synthesis.<n>3DiMo trains a motion encoder with a pretrained video generator to distill driving frames into compact, view-agnostic motion tokens.<n>Experiments confirm that 3DiMo faithfully reproduces driving motions with flexible, text-driven camera control.
  arXiv  Detail & Related papers  (2026-02-03T17:59:09Z)
• N3D-VLM: Native 3D Grounding Enables Accurate Spatial Reasoning in Vision-Language Models [45.008146973701855]
  N3D-VLM is a novel unified framework that seamlessly integrates native 3D object perception with 3D-aware visual reasoning.<n>Unlike conventional end-to-end models that directly predict answers from RGB/RGB-D inputs, our approach equips the model with native 3D object perception capabilities.
  arXiv  Detail & Related papers  (2025-12-18T14:03:44Z)
• Drag4D: Align Your Motion with Text-Driven 3D Scene Generation [77.79131321983677]
  Drag4D is an interactive framework that integrates object motion control within text-driven 3D scene generation.<n>This framework enables users to define 3D trajectories for the 3D objects generated from a single image, seamlessly integrating them into a high-quality 3D background.
  arXiv  Detail & Related papers  (2025-09-26T05:23:45Z)
• 3D Aware Region Prompted Vision Language Model [99.4106711584584]
  SR-3D connects single-view 2D images and multi-view 3D data through a shared visual token space.<n> SR-3D supports flexible region prompting, allowing users to annotate regions with bounding boxes, segmentation masks on any frame, or directly in 3D.
  arXiv  Detail & Related papers  (2025-09-16T17:59:06Z)
• Bridging Diffusion Models and 3D Representations: A 3D Consistent Super-Resolution Framework [51.20764440735875]
  We propose 3D Super Resolution (3DSR), a novel 3D Gaussian-splatting-based super-resolution framework.<n>3DSR encourages 3D consistency across views via the use of an explicit 3D Gaussian-splatting-based scene representation.<n>We evaluate 3DSR on MipNeRF360 and LLFF data, demonstrating that it produces high-resolution results that are visually compelling.
  arXiv  Detail & Related papers  (2025-08-06T05:12:02Z)
• Unifying 2D and 3D Vision-Language Understanding [85.84054120018625]
  We introduce UniVLG, a unified architecture for 2D and 3D vision-language learning.<n>UniVLG bridges the gap between existing 2D-centric models and the rich 3D sensory data available in embodied systems.
  arXiv  Detail & Related papers  (2025-03-13T17:56:22Z)
• 3DTrajMaster: Mastering 3D Trajectory for Multi-Entity Motion in Video Generation [83.98251722144195]
  Previous methods on controllable video generation primarily leverage 2D control signals to manipulate object motions.<n>We introduce 3DTrajMaster, a robust controller that regulates multi-entity dynamics in 3D space.<n>We show that 3DTrajMaster sets a new state-of-the-art in both accuracy and generalization for controlling multi-entity 3D motions.
  arXiv  Detail & Related papers  (2024-12-10T18:55:13Z)
• Lifting Motion to the 3D World via 2D Diffusion [19.64801640086107]
  We introduce MVLift, a novel approach to predict global 3D motion using only 2D pose sequences for training.<n> MVLift generalizes across various domains, including human poses, human-object interactions, and animal poses.
  arXiv  Detail & Related papers  (2024-11-27T23:26:56Z)
• SYM3D: Learning Symmetric Triplanes for Better 3D-Awareness of GANs [5.84660008137615]
  SYM3D is a novel 3D-aware GAN designed to leverage the prevalental symmetry structure found in natural and man-made objects. We demonstrate its superior performance in capturing detailed geometry and texture, even when trained on only single-view images.
  arXiv  Detail & Related papers  (2024-06-10T16:24:07Z)
• Atlas3D: Physically Constrained Self-Supporting Text-to-3D for Simulation and Fabrication [50.541882834405946]
  We introduce Atlas3D, an automatic and easy-to-implement text-to-3D method. Our approach combines a novel differentiable simulation-based loss function with physically inspired regularization. We verify Atlas3D's efficacy through extensive generation tasks and validate the resulting 3D models in both simulated and real-world environments.
  arXiv  Detail & Related papers  (2024-05-28T18:33:18Z)
• RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervision [36.15913507034939]
  We present RenderOcc, a novel paradigm for training 3D occupancy models only using 2D labels. Specifically, we extract a NeRF-style 3D volume representation from multi-view images. We employ volume rendering techniques to establish 2D renderings, thus enabling direct 3D supervision from 2D semantics and depth labels.
  arXiv  Detail & Related papers  (2023-09-18T06:08:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.