A Hierarchical Architecture for Neural Materials

• URL: http://arxiv.org/abs/2307.10135v3
• Date: Wed, 24 Apr 2024 15:07:25 GMT
• Title: A Hierarchical Architecture for Neural Materials
• Authors: Bowen Xue, Shuang Zhao, Henrik Wann Jensen, Zahra Montazeri,
• Abstract summary: We introduce a neural appearance model that offers a new level of accuracy. An inception-based core network structure captures material appearances at multiple scales. We encode the inputs into frequency space, introduce a gradient-based loss, and employ it adaptive to the progress of the learning phase.
• Score: 13.144139872006287
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural reflectance models are capable of reproducing the spatially-varying appearance of many real-world materials at different scales. Unfortunately, existing techniques such as NeuMIP have difficulties handling materials with strong shadowing effects or detailed specular highlights. In this paper, we introduce a neural appearance model that offers a new level of accuracy. Central to our model is an inception-based core network structure that captures material appearances at multiple scales using parallel-operating kernels and ensures multi-stage features through specialized convolution layers. Furthermore, we encode the inputs into frequency space, introduce a gradient-based loss, and employ it adaptive to the progress of the learning phase. We demonstrate the effectiveness of our method using a variety of synthetic and real examples.

Related papers

• Deep Learning Through A Telescoping Lens: A Simple Model Provides Empirical Insights On Grokking, Gradient Boosting & Beyond [61.18736646013446]
  In pursuit of a deeper understanding of its surprising behaviors, we investigate the utility of a simple yet accurate model of a trained neural network. Across three case studies, we illustrate how it can be applied to derive new empirical insights on a diverse range of prominent phenomena.
  arXiv  Detail & Related papers  (2024-10-31T22:54:34Z)
• Approaching Deep Learning through the Spectral Dynamics of Weights [41.948042468042374]
  spectral dynamics of weights -- the behavior of singular values and vectors during optimization -- to clarify and unify several phenomena in deep learning. We identify a consistent bias in optimization across various experiments, from small-scale grokking'' to large-scale tasks like image classification with ConvNets, image generation with UNets, speech recognition with LSTMs, and language modeling with Transformers.
  arXiv  Detail & Related papers  (2024-08-21T17:48:01Z)
• NePF: Neural Photon Field for Single-Stage Inverse Rendering [6.977356702921476]
  We present a novel single-stage framework, Neural Photon Field (NePF), to address the ill-posed inverse rendering from multi-view images. NePF achieves this unification by fully utilizing the physical implication behind the weight function of neural implicit surfaces. We evaluate our method on both real and synthetic datasets.
  arXiv  Detail & Related papers  (2023-11-20T06:15:46Z)
• NeuS-PIR: Learning Relightable Neural Surface using Pre-Integrated Rendering [23.482941494283978]
  This paper presents a method, namely NeuS-PIR, for recovering relightable neural surfaces from multi-view images or video. Unlike methods based on NeRF and discrete meshes, our method utilizes implicit neural surface representation to reconstruct high-quality geometry. Our method enables advanced applications such as relighting, which can be seamlessly integrated with modern graphics engines.
  arXiv  Detail & Related papers  (2023-06-13T09:02:57Z)
• Unsupervised Domain Transfer with Conditional Invertible Neural Networks [83.90291882730925]
  We propose a domain transfer approach based on conditional invertible neural networks (cINNs) Our method inherently guarantees cycle consistency through its invertible architecture, and network training can efficiently be conducted with maximum likelihood. Our method enables the generation of realistic spectral data and outperforms the state of the art on two downstream classification tasks.
  arXiv  Detail & Related papers  (2023-03-17T18:00:27Z)
• Convolutional Learning on Multigraphs [153.20329791008095]
  We develop convolutional information processing on multigraphs and introduce convolutional multigraph neural networks (MGNNs) To capture the complex dynamics of information diffusion within and across each of the multigraph's classes of edges, we formalize a convolutional signal processing model. We develop a multigraph learning architecture, including a sampling procedure to reduce computational complexity. The introduced architecture is applied towards optimal wireless resource allocation and a hate speech localization task, offering improved performance over traditional graph neural networks.
  arXiv  Detail & Related papers  (2022-09-23T00:33:04Z)
• Self-Denoising Neural Networks for Few Shot Learning [66.38505903102373]
  We present a new training scheme that adds noise at multiple stages of an existing neural architecture while simultaneously learning to be robust to this added noise. This architecture, which we call a Self-Denoising Neural Network (SDNN), can be applied easily to most modern convolutional neural architectures.
  arXiv  Detail & Related papers  (2021-10-26T03:28:36Z)
• NeuMIP: Multi-Resolution Neural Materials [98.83749495351627]
  NeuMIP is a neural method for representing and rendering a variety of material appearances at different scales. We generalize traditional mipmap pyramids to pyramids of neural textures, combined with a fully connected network. We also introduce neural offsets, a novel method which allows rendering materials with intricate parallax effects without any tessellation.
  arXiv  Detail & Related papers  (2021-04-06T21:22:22Z)
• A Compact Deep Architecture for Real-time Saliency Prediction [42.58396452892243]
  Saliency models aim to imitate the attention mechanism in the human visual system. Deep models have a high number of parameters which makes them less suitable for real-time applications. Here we propose a compact yet fast model for real-time saliency prediction.
  arXiv  Detail & Related papers  (2020-08-30T17:47:16Z)
• Limited-angle tomographic reconstruction of dense layered objects by dynamical machine learning [68.9515120904028]
  Limited-angle tomography of strongly scattering quasi-transparent objects is a challenging, highly ill-posed problem. Regularizing priors are necessary to reduce artifacts by improving the condition of such problems. We devised a recurrent neural network (RNN) architecture with a novel split-convolutional gated recurrent unit (SC-GRU) as the building block.
  arXiv  Detail & Related papers  (2020-07-21T11:48:22Z)
• Spectrally Consistent UNet for High Fidelity Image Transformations [5.494315657902533]
  Convolutional Neural Networks (CNNs) are the current de-facto models used for many imaging tasks. In this work, a method for assessing the structural biases of UNets and the effects these have on the outputs is presented. A new upsampling module is proposed, based on a novel use of the Guided Image Filter, that provides spectrally consistent outputs.
  arXiv  Detail & Related papers  (2020-04-22T17:04:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.