Towards a Sampling Theory for Implicit Neural Representations

• URL: http://arxiv.org/abs/2405.18410v1
• Date: Tue, 28 May 2024 17:53:47 GMT
• Title: Towards a Sampling Theory for Implicit Neural Representations
• Authors: Mahrokh Najaf, Gregory Ongie,
• Abstract summary: Implicit neural representations (INRs) have emerged as a powerful tool for solving inverse problems in computer and computational imaging. We show how to recover images from a hidden-layer INR using a generalized form of weight decay regularization. We empirically assess the probability of achieving exact recovery images realized by low-width single-layer INRs, and illustrate the performance of INR on super-resolution recovery of more realistic continuous domain phantom images.
• Score: 0.3222802562733786
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Implicit neural representations (INRs) have emerged as a powerful tool for solving inverse problems in computer vision and computational imaging. INRs represent images as continuous domain functions realized by a neural network taking spatial coordinates as inputs. However, unlike traditional pixel representations, little is known about the sample complexity of estimating images using INRs in the context of linear inverse problems. Towards this end, we study the sampling requirements for recovery of a continuous domain image from its low-pass Fourier coefficients by fitting a single hidden-layer INR with ReLU activation and a Fourier features layer using a generalized form of weight decay regularization. Our key insight is to relate minimizers of this non-convex parameter space optimization problem to minimizers of a convex penalty defined over an infinite-dimensional space of measures. We identify a sufficient number of samples for which an image realized by a width-1 INR is exactly recoverable by solving the INR training problem, and give a conjecture for the general width-$W$ case. To validate our theory, we empirically assess the probability of achieving exact recovery of images realized by low-width single hidden-layer INRs, and illustrate the performance of INR on super-resolution recovery of more realistic continuous domain phantom images.

Related papers

• Streaming Neural Images [56.41827271721955]
  Implicit Neural Representations (INRs) are a novel paradigm for signal representation that have attracted considerable interest for image compression. In this work, we explore the critical yet overlooked limiting factors of INRs, such as computational cost, unstable performance, and robustness.
  arXiv  Detail & Related papers  (2024-09-25T17:51:20Z)
• BID-NeRF: RGB-D image pose estimation with inverted Neural Radiance Fields [0.0]
  We aim to improve the Inverted Neural Radiance Fields (iNeRF) algorithm which defines the image pose estimation problem as a NeRF based iterative linear optimization. NeRFs are novel neural space representation models that can synthesize photorealistic novel views of real-world scenes or objects.
  arXiv  Detail & Related papers  (2023-10-05T14:27:06Z)
• Sobolev Training for Implicit Neural Representations with Approximated Image Derivatives [12.71676484494428]
  Implicit Neural Representations (INRs) parameterized by neural networks have emerged as a powerful tool to represent different kinds of signals. We propose a training paradigm for INRs whose target output is image pixels, to encode image derivatives in addition to image values in the neural network. We show how the training paradigm can be leveraged to solve typical INRs problems, i.e., image regression and inverse rendering.
  arXiv  Detail & Related papers  (2022-07-21T10:12:41Z)
• Denoising Diffusion Restoration Models [110.1244240726802]
  Denoising Diffusion Restoration Models (DDRM) is an efficient, unsupervised posterior sampling method. We demonstrate DDRM's versatility on several image datasets for super-resolution, deblurring, inpainting, and colorization.
  arXiv  Detail & Related papers  (2022-01-27T20:19:07Z)
• InfoNeRF: Ray Entropy Minimization for Few-Shot Neural Volume Rendering [55.70938412352287]
  We present an information-theoretic regularization technique for few-shot novel view synthesis based on neural implicit representation. The proposed approach minimizes potential reconstruction inconsistency that happens due to insufficient viewpoints. We achieve consistently improved performance compared to existing neural view synthesis methods by large margins on multiple standard benchmarks.
  arXiv  Detail & Related papers  (2021-12-31T11:56:01Z)
• IREM: High-Resolution Magnetic Resonance (MR) Image Reconstruction via Implicit Neural Representation [33.55719364798433]
  We propose a novel image reconstruction network named IREM, which is trained on multiple low-resolution (LR) MR images. IREM reduces scan time and achieves high-resolution MR imaging in terms of SNR and local image detail.
  arXiv  Detail & Related papers  (2021-06-29T05:25:43Z)
• Deep Unfolded Recovery of Sub-Nyquist Sampled Ultrasound Image [94.42139459221784]
  We propose a reconstruction method from sub-Nyquist samples in the time and spatial domain, that is based on unfolding the ISTA algorithm. Our method allows reducing the number of array elements, sampling rate, and computational time while ensuring high quality imaging performance.
  arXiv  Detail & Related papers  (2021-03-01T19:19:38Z)
• Learned convex regularizers for inverse problems [3.294199808987679]
  We propose to learn a data-adaptive input- neural network (ICNN) as a regularizer for inverse problems. We prove the existence of a sub-gradient-based algorithm that leads to a monotonically decreasing error in the parameter space with iterations. We show that the proposed convex regularizer is at least competitive with and sometimes superior to state-of-the-art data-driven techniques for inverse problems.
  arXiv  Detail & Related papers  (2020-08-06T18:58:35Z)
• Cross-Scale Internal Graph Neural Network for Image Super-Resolution [147.77050877373674]
  Non-local self-similarity in natural images has been well studied as an effective prior in image restoration. For single image super-resolution (SISR), most existing deep non-local methods only exploit similar patches within the same scale of the low-resolution (LR) input image. This is achieved using a novel cross-scale internal graph neural network (IGNN)
  arXiv  Detail & Related papers  (2020-06-30T10:48:40Z)
• Hyperspectral Image Super-resolution via Deep Progressive Zero-centric Residual Learning [62.52242684874278]
  Cross-modality distribution of spatial and spectral information makes the problem challenging. We propose a novel textitlightweight deep neural network-based framework, namely PZRes-Net. Our framework learns a high resolution and textitzero-centric residual image, which contains high-frequency spatial details of the scene.
  arXiv  Detail & Related papers  (2020-06-18T06:32:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.