Neural BRDF Representation and Importance Sampling
- URL: http://arxiv.org/abs/2102.05963v2
- Date: Fri, 12 Feb 2021 12:38:18 GMT
- Title: Neural BRDF Representation and Importance Sampling
- Authors: Alejandro Sztrajman, Gilles Rainer, Tobias Ritschel, Tim Weyrich
- Abstract summary: We present a compact neural network-based representation of reflectance BRDF data.
We encode BRDFs as lightweight networks, and propose a training scheme with adaptive angular sampling.
We evaluate encoding results on isotropic and anisotropic BRDFs from multiple real-world datasets.
- Score: 79.84316447473873
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Controlled capture of real-world material appearance yields tabulated sets of
highly realistic reflectance data. In practice, however, its high memory
footprint requires compressing into a representation that can be used
efficiently in rendering while remaining faithful to the original. Previous
works in appearance encoding often prioritised one of these requirements at the
expense of the other, by either applying high-fidelity array compression
strategies not suited for efficient queries during rendering, or by fitting a
compact analytic model that lacks expressiveness. We present a compact neural
network-based representation of BRDF data that combines high-accuracy
reconstruction with efficient practical rendering via built-in interpolation of
reflectance. We encode BRDFs as lightweight networks, and propose a training
scheme with adaptive angular sampling, critical for the accurate reconstruction
of specular highlights. Additionally, we propose a novel approach to make our
representation amenable to importance sampling: rather than inverting the
trained networks, we learn an embedding that can be mapped to parameters of an
analytic BRDF for which importance sampling is known. We evaluate encoding
results on isotropic and anisotropic BRDFs from multiple real-world datasets,
and importance sampling performance for isotropic BRDFs mapped to two different
analytic models.
Related papers
- Low-Rank Representations Meets Deep Unfolding: A Generalized and
Interpretable Network for Hyperspectral Anomaly Detection [41.50904949744355]
Current hyperspectral anomaly detection (HAD) benchmark datasets suffer from low resolution, simple background, and small size of the detection data.
These factors also limit the performance of the well-known low-rank representation (LRR) models in terms of robustness.
We build a new set of HAD benchmark datasets for improving the robustness of the HAD algorithm in complex scenarios, AIR-HAD for short.
arXiv Detail & Related papers (2024-02-23T14:15:58Z) - Learning Image Deraining Transformer Network with Dynamic Dual
Self-Attention [46.11162082219387]
This paper proposes an effective image deraining Transformer with dynamic dual self-attention (DDSA)
Specifically, we only select the most useful similarity values based on top-k approximate calculation to achieve sparse attention.
In addition, we also develop a novel spatial-enhanced feed-forward network (SEFN) to further obtain a more accurate representation for achieving high-quality derained results.
arXiv Detail & Related papers (2023-08-15T13:59:47Z) - End-to-End Meta-Bayesian Optimisation with Transformer Neural Processes [52.818579746354665]
This paper proposes the first end-to-end differentiable meta-BO framework that generalises neural processes to learn acquisition functions via transformer architectures.
We enable this end-to-end framework with reinforcement learning (RL) to tackle the lack of labelled acquisition data.
arXiv Detail & Related papers (2023-05-25T10:58:46Z) - AdaNeRF: Adaptive Sampling for Real-time Rendering of Neural Radiance
Fields [8.214695794896127]
Novel view synthesis has recently been revolutionized by learning neural radiance fields directly from sparse observations.
rendering images with this new paradigm is slow due to the fact that an accurate quadrature of the volume rendering equation requires a large number of samples for each ray.
We propose a novel dual-network architecture that takes an direction by learning how to best reduce the number of required sample points.
arXiv Detail & Related papers (2022-07-21T05:59:13Z) - Transformer-based Context Condensation for Boosting Feature Pyramids in
Object Detection [77.50110439560152]
Current object detectors typically have a feature pyramid (FP) module for multi-level feature fusion (MFF)
We propose a novel and efficient context modeling mechanism that can help existing FPs deliver better MFF results.
In particular, we introduce a novel insight that comprehensive contexts can be decomposed and condensed into two types of representations for higher efficiency.
arXiv Detail & Related papers (2022-07-14T01:45:03Z) - CSformer: Bridging Convolution and Transformer for Compressive Sensing [65.22377493627687]
This paper proposes a hybrid framework that integrates the advantages of leveraging detailed spatial information from CNN and the global context provided by transformer for enhanced representation learning.
The proposed approach is an end-to-end compressive image sensing method, composed of adaptive sampling and recovery.
The experimental results demonstrate the effectiveness of the dedicated transformer-based architecture for compressive sensing.
arXiv Detail & Related papers (2021-12-31T04:37:11Z) - NeRF in detail: Learning to sample for view synthesis [104.75126790300735]
Neural radiance fields (NeRF) methods have demonstrated impressive novel view synthesis.
In this work we address a clear limitation of the vanilla coarse-to-fine approach -- that it is based on a performance and not trained end-to-end for the task at hand.
We introduce a differentiable module that learns to propose samples and their importance for the fine network, and consider and compare multiple alternatives for its neural architecture.
arXiv Detail & Related papers (2021-06-09T17:59:10Z) - Invertible Neural BRDF for Object Inverse Rendering [27.86441556552318]
We introduce a novel neural network-based BRDF model and a Bayesian framework for object inverse rendering.
We experimentally validate the accuracy of the invertible neural BRDF model on a large number of measured data.
Results show new ways in which deep neural networks can help solve challenging inverse problems.
arXiv Detail & Related papers (2020-08-10T11:27:01Z) - Resolution Adaptive Networks for Efficient Inference [53.04907454606711]
We propose a novel Resolution Adaptive Network (RANet), which is inspired by the intuition that low-resolution representations are sufficient for classifying "easy" inputs.
In RANet, the input images are first routed to a lightweight sub-network that efficiently extracts low-resolution representations.
High-resolution paths in the network maintain the capability to recognize the "hard" samples.
arXiv Detail & Related papers (2020-03-16T16:54:36Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.