NOVA-3D: Non-overlapped Views for 3D Anime Character Reconstruction

• URL: http://arxiv.org/abs/2405.12505v1
• Date: Tue, 21 May 2024 05:31:03 GMT
• Title: NOVA-3D: Non-overlapped Views for 3D Anime Character Reconstruction
• Authors: Hongsheng Wang, Nanjie Yao, Xinrui Zhou, Shengyu Zhang, Huahao Xu, Fei Wu, Feng Lin,
• Abstract summary: Non-Overlapped Views for 3D textbfAnime Character Reconstruction (NOVA-3D) New framework implements a method for view-aware feature fusion to learn 3D-consistent features effectively. Experiments demonstrate superior reconstruction of anime characters with exceptional detail fidelity.
• Score: 14.509202872426942
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the animation industry, 3D modelers typically rely on front and back non-overlapped concept designs to guide the 3D modeling of anime characters. However, there is currently a lack of automated approaches for generating anime characters directly from these 2D designs. In light of this, we explore a novel task of reconstructing anime characters from non-overlapped views. This presents two main challenges: existing multi-view approaches cannot be directly applied due to the absence of overlapping regions, and there is a scarcity of full-body anime character data and standard benchmarks. To bridge the gap, we present Non-Overlapped Views for 3D \textbf{A}nime Character Reconstruction (NOVA-3D), a new framework that implements a method for view-aware feature fusion to learn 3D-consistent features effectively and synthesizes full-body anime characters from non-overlapped front and back views directly. To facilitate this line of research, we collected the NOVA-Human dataset, which comprises multi-view images and accurate camera parameters for 3D anime characters. Extensive experiments demonstrate that the proposed method outperforms baseline approaches, achieving superior reconstruction of anime characters with exceptional detail fidelity. In addition, to further verify the effectiveness of our method, we applied it to the animation head reconstruction task and improved the state-of-the-art baseline to 94.453 in SSIM, 7.726 in LPIPS, and 19.575 in PSNR on average. Codes and datasets are available at https://wanghongsheng01.github.io/NOVA-3D/.

Related papers

• Toon3D: Seeing Cartoons from a New Perspective [52.85312338932685]
  We focus our analysis on hand-drawn images from cartoons and anime. Many cartoons are created by artists without a 3D rendering engine, which means that any new image of a scene is hand-drawn. We correct for 2D drawing inconsistencies to recover a plausible 3D structure such that the newly warped drawings are consistent with each other.
  arXiv  Detail & Related papers  (2024-05-16T17:59:51Z)
• Agent3D-Zero: An Agent for Zero-shot 3D Understanding [79.88440434836673]
  Agent3D-Zero is an innovative 3D-aware agent framework addressing the 3D scene understanding. We propose a novel way to make use of a Large Visual Language Model (VLM) via actively selecting and analyzing a series of viewpoints for 3D understanding. A distinctive advantage of Agent3D-Zero is the introduction of novel visual prompts, which significantly unleash the VLMs' ability to identify the most informative viewpoints.
  arXiv  Detail & Related papers  (2024-03-18T14:47:03Z)
• PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigm [114.47216525866435]
  We introduce a novel universal 3D pre-training framework designed to facilitate the acquisition of efficient 3D representation. For the first time, PonderV2 achieves state-of-the-art performance on 11 indoor and outdoor benchmarks, implying its effectiveness.
  arXiv  Detail & Related papers  (2023-10-12T17:59:57Z)
• PAniC-3D: Stylized Single-view 3D Reconstruction from Portraits of Anime Characters [29.107457721261387]
  We propose PAniC-3D, a system to reconstruct stylized 3D character heads directly from illustrated (p)ortraits of (ani)me (c)haracters.
  arXiv  Detail & Related papers  (2023-03-25T23:36:17Z)
• SHERF: Generalizable Human NeRF from a Single Image [59.10589479808622]
  SHERF is the first generalizable Human NeRF model for recovering animatable 3D humans from a single input image. We propose a bank of 3D-aware hierarchical features, including global, point-level, and pixel-aligned features, to facilitate informative encoding.
  arXiv  Detail & Related papers  (2023-03-22T17:59:12Z)
• RaBit: Parametric Modeling of 3D Biped Cartoon Characters with a Topological-consistent Dataset [19.494054103293116]
  We introduce 3DBiCar, the first large-scale dataset of 3D biped cartoon characters, and RaBit, the corresponding parametric model. RaBit is built with a SMPL-like linear blend shape model and a StyleGAN-based neural UV-texture generator, simultaneously expressing the shape, pose, and texture. Various applications are conducted, including single-view reconstruction, sketch-based modeling, and 3D cartoon animation.
  arXiv  Detail & Related papers  (2023-03-22T13:46:15Z)
• AnimeRun: 2D Animation Visual Correspondence from Open Source 3D Movies [98.65469430034246]
  Existing datasets for two-dimensional (2D) cartoon suffer from simple frame composition and monotonic movements. We present a new 2D animation visual correspondence dataset, AnimeRun, by converting open source 3D movies to full scenes in 2D style. Our analyses show that the proposed dataset not only resembles real anime more in image composition, but also possesses richer and more complex motion patterns compared to existing datasets.
  arXiv  Detail & Related papers  (2022-11-10T17:26:21Z)
• DRaCoN -- Differentiable Rasterization Conditioned Neural Radiance Fields for Articulated Avatars [92.37436369781692]
  We present DRaCoN, a framework for learning full-body volumetric avatars. It exploits the advantages of both the 2D and 3D neural rendering techniques. Experiments on the challenging ZJU-MoCap and Human3.6M datasets indicate that DRaCoN outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-03-29T17:59:15Z)
• ARCH: Animatable Reconstruction of Clothed Humans [27.849315613277724]
  ARCH (Animatable Reconstruction of Clothed Humans) is an end-to-end framework for accurate reconstruction of animation-ready 3D clothed humans from a monocular image. ARCH is a learned pose-aware model that produces detailed 3D rigged full-body human avatars from a single unconstrained RGB image.
  arXiv  Detail & Related papers  (2020-04-08T14:23:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.