ANR: Articulated Neural Rendering for Virtual Avatars
- URL: http://arxiv.org/abs/2012.12890v1
- Date: Wed, 23 Dec 2020 18:56:11 GMT
- Title: ANR: Articulated Neural Rendering for Virtual Avatars
- Authors: Amit Raj, Julian Tanke, James Hays, Minh Vo, Carsten Stoll, Christoph
Lassner
- Abstract summary: We present Articulated Neural Rendering (ANR), a novel framework based on Deferred Neural Rendering (DNR)
We show the superiority of ANR not only with respect to DNR but also with methods specialized for avatar creation and animation.
- Score: 30.787360376175123
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The combination of traditional rendering with neural networks in Deferred
Neural Rendering (DNR) provides a compelling balance between computational
complexity and realism of the resulting images. Using skinned meshes for
rendering articulating objects is a natural extension for the DNR framework and
would open it up to a plethora of applications. However, in this case the
neural shading step must account for deformations that are possibly not
captured in the mesh, as well as alignment inaccuracies and dynamics -- which
can confound the DNR pipeline. We present Articulated Neural Rendering (ANR), a
novel framework based on DNR which explicitly addresses its limitations for
virtual human avatars. We show the superiority of ANR not only with respect to
DNR but also with methods specialized for avatar creation and animation. In two
user studies, we observe a clear preference for our avatar model and we
demonstrate state-of-the-art performance on quantitative evaluation metrics.
Perceptually, we observe better temporal stability, level of detail and
plausibility.
Related papers
- Towards a Sampling Theory for Implicit Neural Representations [0.3222802562733786]
Implicit neural representations (INRs) have emerged as a powerful tool for solving inverse problems in computer and computational imaging.
We show how to recover images from a hidden-layer INR using a generalized form of weight decay regularization.
We empirically assess the probability of achieving exact recovery images realized by low-width single-layer INRs, and illustrate the performance of INR on super-resolution recovery of more realistic continuous domain phantom images.
arXiv Detail & Related papers (2024-05-28T17:53:47Z) - Revisiting Implicit Neural Representations in Low-Level Vision [20.3578908524788]
Implicit Neural Representation (INR) has been emerging in computer vision in recent years.
We are interested in its effectiveness in low-level vision problems such as image restoration.
In this work, we revisit INR and investigate its application in low-level image restoration tasks.
arXiv Detail & Related papers (2023-04-20T12:19:27Z) - DNeRV: Modeling Inherent Dynamics via Difference Neural Representation
for Videos [53.077189668346705]
Difference Representation for Videos (eRV)
We analyze this from the perspective of limitation function fitting and the importance of frame difference.
DNeRV achieves competitive results against the state-of-the-art neural compression approaches.
arXiv Detail & Related papers (2023-04-13T13:53:49Z) - WIRE: Wavelet Implicit Neural Representations [42.147899723673596]
Implicit neural representations (INRs) have recently advanced numerous vision-related areas.
Current INRs designed to have high accuracy also suffer from poor robustness.
We develop a new, highly accurate and robust INR that does not exhibit this tradeoff.
arXiv Detail & Related papers (2023-01-05T20:24:56Z) - Signal Processing for Implicit Neural Representations [80.38097216996164]
Implicit Neural Representations (INRs) encode continuous multi-media data via multi-layer perceptrons.
Existing works manipulate such continuous representations via processing on their discretized instance.
We propose an implicit neural signal processing network, dubbed INSP-Net, via differential operators on INR.
arXiv Detail & Related papers (2022-10-17T06:29:07Z) - Neural Adaptive SCEne Tracing [24.781844909539686]
We present NAScenT, the first neural rendering method based on directly training a hybrid explicit-implicit neural representation.
NAScenT is capable of reconstructing challenging scenes including both large, sparsely populated volumes like UAV captured outdoor environments.
arXiv Detail & Related papers (2022-02-28T10:27:23Z) - InfoNeRF: Ray Entropy Minimization for Few-Shot Neural Volume Rendering [55.70938412352287]
We present an information-theoretic regularization technique for few-shot novel view synthesis based on neural implicit representation.
The proposed approach minimizes potential reconstruction inconsistency that happens due to insufficient viewpoints.
We achieve consistently improved performance compared to existing neural view synthesis methods by large margins on multiple standard benchmarks.
arXiv Detail & Related papers (2021-12-31T11:56:01Z) - Towards Lightweight Controllable Audio Synthesis with Conditional
Implicit Neural Representations [10.484851004093919]
Implicit neural representations (INRs) are neural networks used to approximate low-dimensional functions.
In this work we shed light on the potential of Conditional Implicit Neural Representations (CINRs) as lightweight backbones in generative frameworks for audio synthesis.
arXiv Detail & Related papers (2021-11-14T13:36:18Z) - Neural Rays for Occlusion-aware Image-based Rendering [108.34004858785896]
We present a new neural representation, called Neural Ray (NeuRay), for the novel view synthesis (NVS) task with multi-view images as input.
NeuRay can quickly generate high-quality novel view rendering images of unseen scenes with little finetuning.
arXiv Detail & Related papers (2021-07-28T15:09:40Z) - UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for
Multi-View Reconstruction [61.17219252031391]
We present a novel method for reconstructing surfaces from multi-view images using Neural implicit 3D representations.
Our key insight is that implicit surface models and radiance fields can be formulated in a unified way, enabling both surface and volume rendering.
Our experiments demonstrate that we outperform NeRF in terms of reconstruction quality while performing on par with IDR without requiring masks.
arXiv Detail & Related papers (2021-04-20T15:59:38Z) - Variational Denoising Network: Toward Blind Noise Modeling and Removal [59.36166491196973]
Blind image denoising is an important yet very challenging problem in computer vision.
We propose a new variational inference method, which integrates both noise estimation and image denoising.
arXiv Detail & Related papers (2019-08-29T15:54:06Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.