Deblur-Avatar: Animatable Avatars from Motion-Blurred Monocular Videos

• URL: http://arxiv.org/abs/2501.13335v1
• Date: Thu, 23 Jan 2025 02:31:57 GMT
• Title: Deblur-Avatar: Animatable Avatars from Motion-Blurred Monocular Videos
• Authors: Xianrui Luo, Juewen Peng, Zhongang Cai, Lei Yang, Fan Yang, Zhiguo Cao, Guosheng Lin,
• Abstract summary: De-Avatar is a novel framework for modeling high-fidelity, animatable 3D human avatars from motion-blurred monocular video inputs.
• Score: 64.10307207290039
• License:
• Abstract: We introduce Deblur-Avatar, a novel framework for modeling high-fidelity, animatable 3D human avatars from motion-blurred monocular video inputs. Motion blur is prevalent in real-world dynamic video capture, especially due to human movements in 3D human avatar modeling. Existing methods either (1) assume sharp image inputs, failing to address the detail loss introduced by motion blur, or (2) mainly consider blur by camera movements, neglecting the human motion blur which is more common in animatable avatars. Our proposed approach integrates a human movement-based motion blur model into 3D Gaussian Splatting (3DGS). By explicitly modeling human motion trajectories during exposure time, we jointly optimize the trajectories and 3D Gaussians to reconstruct sharp, high-quality human avatars. We employ a pose-dependent fusion mechanism to distinguish moving body regions, optimizing both blurred and sharp areas effectively. Extensive experiments on synthetic and real-world datasets demonstrate that Deblur-Avatar significantly outperforms existing methods in rendering quality and quantitative metrics, producing sharp avatar reconstructions and enabling real-time rendering under challenging motion blur conditions.

Related papers

• Move-in-2D: 2D-Conditioned Human Motion Generation [54.067588636155115]
  We propose Move-in-2D, a novel approach to generate human motion sequences conditioned on a scene image. Our approach accepts both a scene image and text prompt as inputs, producing a motion sequence tailored to the scene.
  arXiv  Detail & Related papers  (2024-12-17T18:58:07Z)
• Bundle Adjusted Gaussian Avatars Deblurring [31.718130377229482]
  We propose a 3D-aware, physics-oriented model of blur formation attributable to human movement and a 3D human motion model to clarify ambiguities found in motion-induced blurry images. We have established benchmarks for this task through a synthetic dataset derived from existing multi-view captures, alongside a real-captured dataset acquired through a 360-degree synchronous hybrid-exposure camera system.
  arXiv  Detail & Related papers  (2024-11-24T10:03:24Z)
• AvatarGO: Zero-shot 4D Human-Object Interaction Generation and Animation [60.5897687447003]
  AvatarGO is a novel framework designed to generate realistic 4D HOI scenes from textual inputs. Our framework not only generates coherent compositional motions, but also exhibits greater robustness in handling issues. As the first attempt to synthesize 4D avatars with object interactions, we hope AvatarGO could open new doors for human-centric 4D content creation.
  arXiv  Detail & Related papers  (2024-10-09T17:58:56Z)
• ExFMan: Rendering 3D Dynamic Humans with Hybrid Monocular Blurry Frames and Events [7.820081911598502]
  We propose ExFMan, the first neural rendering framework that renders high-quality humans in rapid motion with a hybrid frame-based RGB and bio-inspired event camera. We first formulate a velocity field of the 3D body in the canonical space and render it to image space to identify the body parts with motion blur. We then propose two novel losses, i.e., velocity-aware photometric loss and velocity-relative event loss, to optimize the neural human for both modalities.
  arXiv  Detail & Related papers  (2024-09-21T10:58:01Z)
• Motion Avatar: Generate Human and Animal Avatars with Arbitrary Motion [39.456643736018435]
  We propose a novel agent-based approach named Motion Avatar, which allows for the automatic generation of high-quality customizable human and animal avatars. Secondly, we introduced a LLM planner that coordinates both motion and avatar generation, which transforms a discriminative planning into a customizable Q&A fashion. Finally, we presented an animal motion dataset named Zoo-300K, comprising approximately 300,000 text-motion pairs across 65 animal categories.
  arXiv  Detail & Related papers  (2024-05-18T13:21:14Z)
• GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussians [51.46168990249278]
  We present an efficient approach to creating realistic human avatars with dynamic 3D appearances from a single video. GustafAvatar is validated on both the public dataset and our collected dataset.
  arXiv  Detail & Related papers  (2023-12-04T18:55:45Z)
• Decoupling Human and Camera Motion from Videos in the Wild [67.39432972193929]
  We propose a method to reconstruct global human trajectories from videos in the wild. Our method decouples the camera and human motion, which allows us to place people in the same world coordinate frame.
  arXiv  Detail & Related papers  (2023-02-24T18:59:15Z)
• HULC: 3D Human Motion Capture with Pose Manifold Sampling and Dense Contact Guidance [82.09463058198546]
  Marker-less monocular 3D human motion capture (MoCap) with scene interactions is a challenging research topic relevant for extended reality, robotics and virtual avatar generation. We propose HULC, a new approach for 3D human MoCap which is aware of the scene geometry.
  arXiv  Detail & Related papers  (2022-05-11T17:59:31Z)
• Action2video: Generating Videos of Human 3D Actions [31.665831044217363]
  We aim to tackle the interesting yet challenging problem of generating videos of diverse and natural human motions from prescribed action categories. Key issue lies in the ability to synthesize multiple distinct motion sequences that are realistic in their visual appearances. Action2motionally generates plausible 3D pose sequences of a prescribed action category, which are processed and rendered by motion2video to form 2D videos.
  arXiv  Detail & Related papers  (2021-11-12T20:20:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.