Recovering Dynamic 3D Sketches from Videos

• URL: http://arxiv.org/abs/2503.20321v2
• Date: Thu, 27 Mar 2025 06:31:57 GMT
• Title: Recovering Dynamic 3D Sketches from Videos
• Authors: Jaeah Lee, Changwoon Choi, Young Min Kim, Jaesik Park,
• Abstract summary: Liv3Stroke is a novel approach for abstracting objects in motion with deformable 3D strokes.<n>We first extract noisy, 3D point cloud motion guidance from video frames using semantic features.<n>Our approach deforms a set of curves to abstract essential motion features as a set of explicit 3D representations.
• Score: 30.87733869892925
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Understanding 3D motion from videos presents inherent challenges due to the diverse types of movement, ranging from rigid and deformable objects to articulated structures. To overcome this, we propose Liv3Stroke, a novel approach for abstracting objects in motion with deformable 3D strokes. The detailed movements of an object may be represented by unstructured motion vectors or a set of motion primitives using a pre-defined articulation from a template model. Just as a free-hand sketch can intuitively visualize scenes or intentions with a sparse set of lines, we utilize a set of parametric 3D curves to capture a set of spatially smooth motion elements for general objects with unknown structures. We first extract noisy, 3D point cloud motion guidance from video frames using semantic features, and our approach deforms a set of curves to abstract essential motion features as a set of explicit 3D representations. Such abstraction enables an understanding of prominent components of motions while maintaining robustness to environmental factors. Our approach allows direct analysis of 3D object movements from video, tackling the uncertainty that typically occurs when translating real-world motion into recorded footage. The project page is accessible via: https://jaeah.me/liv3stroke_web

Related papers

• In-2-4D: Inbetweening from Two Single-View Images to 4D Generation [54.62824686338408]
  We propose a new problem, In-between2-4D, for generative 4D (i.e., 3D + motion) in Splating from a minimalistic input setting. Given two images representing the start and end states of an object in motion, our goal is to generate and reconstruct the motion in 4D.
  arXiv  Detail & Related papers  (2025-04-11T09:01:09Z)
• Articulate That Object Part (ATOP): 3D Part Articulation via Text and Motion Personalization [9.231848716070257]
  ATOP (Articulate That Object Part) is a novel few-shot method based on motion personalization to articulate a static 3D object.<n>We show that our method is capable of generating realistic motion videos and predicting 3D motion parameters in a more accurate and generalizable way.
  arXiv  Detail & Related papers  (2025-02-11T05:47:16Z)
• Shape of Motion: 4D Reconstruction from a Single Video [51.04575075620677]
  We introduce a method capable of reconstructing generic dynamic scenes, featuring explicit, full-sequence-long 3D motion. We exploit the low-dimensional structure of 3D motion by representing scene motion with a compact set of SE3 motion bases. Our method achieves state-of-the-art performance for both long-range 3D/2D motion estimation and novel view synthesis on dynamic scenes.
  arXiv  Detail & Related papers  (2024-07-18T17:59:08Z)
• DreamScene4D: Dynamic Multi-Object Scene Generation from Monocular Videos [21.93514516437402]
  We present DreamScene4D, the first approach to generate 3D dynamic scenes of multiple objects from monocular videos via novel view synthesis. Our key insight is a "decompose-recompose" approach that factorizes the video scene into the background and object tracks. We show extensive results on challenging DAVIS, Kubric, and self-captured videos with quantitative comparisons and a user preference study.
  arXiv  Detail & Related papers  (2024-05-03T17:55:34Z)
• CAMM: Building Category-Agnostic and Animatable 3D Models from Monocular Videos [3.356334042188362]
  We propose a novel reconstruction method that learns an animatable kinematic chain for any articulated object. Our approach is on par with state-of-the-art 3D surface reconstruction methods on various articulated object categories.
  arXiv  Detail & Related papers  (2023-04-14T06:07:54Z)
• Generating Continual Human Motion in Diverse 3D Scenes [51.90506920301473]
  We introduce a method to synthesize animator guided human motion across 3D scenes. We decompose the continual motion synthesis problem into walking along paths and transitioning in and out of the actions specified by the keypoints. Our model can generate long sequences of diverse actions such as grabbing, sitting and leaning chained together.
  arXiv  Detail & Related papers  (2023-04-04T18:24:22Z)
• Neural Groundplans: Persistent Neural Scene Representations from a Single Image [90.04272671464238]
  We present a method to map 2D image observations of a scene to a persistent 3D scene representation. We propose conditional neural groundplans as persistent and memory-efficient scene representations.
  arXiv  Detail & Related papers  (2022-07-22T17:41:24Z)
• MoCaNet: Motion Retargeting in-the-wild via Canonicalization Networks [77.56526918859345]
  We present a novel framework that brings the 3D motion task from controlled environments to in-the-wild scenarios. It is capable of body motion from a character in a 2D monocular video to a 3D character without using any motion capture system or 3D reconstruction procedure.
  arXiv  Detail & Related papers  (2021-12-19T07:52:05Z)
• Motion-from-Blur: 3D Shape and Motion Estimation of Motion-blurred Objects in Videos [115.71874459429381]
  We propose a method for jointly estimating the 3D motion, 3D shape, and appearance of highly motion-blurred objects from a video. Experiments on benchmark datasets demonstrate that our method outperforms previous methods for fast moving object deblurring and 3D reconstruction.
  arXiv  Detail & Related papers  (2021-11-29T11:25:14Z)
• NeuralDiff: Segmenting 3D objects that move in egocentric videos [92.95176458079047]
  We study the problem of decomposing the observed 3D scene into a static background and a dynamic foreground. This task is reminiscent of the classic background subtraction problem, but is significantly harder because all parts of the scene, static and dynamic, generate a large apparent motion. In particular, we consider egocentric videos and further separate the dynamic component into objects and the actor that observes and moves them.
  arXiv  Detail & Related papers  (2021-10-19T12:51:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.