Predicting the Encoding Error of SIRENs

• URL: http://arxiv.org/abs/2410.21645v1
• Date: Tue, 29 Oct 2024 01:19:22 GMT
• Title: Predicting the Encoding Error of SIRENs
• Authors: Jeremy Vonderfecht, Feng Liu,
• Abstract summary: Implicit Neural Representations (INRs) encode signals such as images, videos, and 3D shapes in the weights of neural networks. We present a method which predicts the encoding error that a popular INR network (SIREN) will reach.
• Score: 4.673285689826945
• License:
• Abstract: Implicit Neural Representations (INRs), which encode signals such as images, videos, and 3D shapes in the weights of neural networks, are becoming increasingly popular. Among their many applications is signal compression, for which there is great interest in achieving the highest possible fidelity to the original signal subject to constraints such as neural network size, training (encoding) and inference (decoding) time. But training INRs can be a computationally expensive process, making it challenging to determine the best possible tradeoff under such constraints. Towards this goal, we present a method which predicts the encoding error that a popular INR network (SIREN) will reach, given its network hyperparameters and the signal to encode. This method is trained on a unique dataset of 300,000 SIRENs, trained across a variety of images and hyperparameters. (Dataset available here: https://huggingface.co/datasets/predict-SIREN-PSNR/COIN-collection.) Our predictive method demonstrates the feasibility of this regression problem, and allows users to anticipate the encoding error that a SIREN network will reach in milliseconds instead of minutes or longer. We also provide insights into the behavior of SIREN networks, such as why narrow SIRENs can have very high random variation in encoding error, and how the performance of SIRENs relates to JPEG compression.

Related papers

• Towards Croppable Implicit Neural Representations [9.372436024276828]
  Implicit Neural Representations (INRs) have peaked interest in recent years due to their ability to encode natural signals using neural networks. In this paper we explore the idea of editable INRs, and specifically focus on the widely used cropping operation. We present Local-Global SIRENs -- a novel INR architecture that supports cropping by design.
  arXiv  Detail & Related papers  (2024-09-28T22:41:49Z)
• INCODE: Implicit Neural Conditioning with Prior Knowledge Embeddings [4.639495398851869]
  Implicit Neural Representations (INRs) have revolutionized signal representation by leveraging neural networks to provide continuous and smooth representations of complex data. We introduce INCODE, a novel approach that enhances the control of the sinusoidal-based activation function in INRs using deep prior knowledge. Our approach not only excels in representation, but also extends its prowess to tackle complex tasks such as audio, image, and 3D shape reconstructions.
  arXiv  Detail & Related papers  (2023-10-28T23:16:49Z)
• DINER: Disorder-Invariant Implicit Neural Representation [33.10256713209207]
  Implicit neural representation (INR) characterizes the attributes of a signal as a function of corresponding coordinates. We propose the disorder-invariant implicit neural representation (DINER) by augmenting a hash-table to a traditional INR backbone.
  arXiv  Detail & Related papers  (2022-11-15T03:34:24Z)
• Signal Processing for Implicit Neural Representations [80.38097216996164]
  Implicit Neural Representations (INRs) encode continuous multi-media data via multi-layer perceptrons. Existing works manipulate such continuous representations via processing on their discretized instance. We propose an implicit neural signal processing network, dubbed INSP-Net, via differential operators on INR.
  arXiv  Detail & Related papers  (2022-10-17T06:29:07Z)
• NAF: Neural Attenuation Fields for Sparse-View CBCT Reconstruction [79.13750275141139]
  This paper proposes a novel and fast self-supervised solution for sparse-view CBCT reconstruction. The desired attenuation coefficients are represented as a continuous function of 3D spatial coordinates, parameterized by a fully-connected deep neural network. A learning-based encoder entailing hash coding is adopted to help the network capture high-frequency details.
  arXiv  Detail & Related papers  (2022-09-29T04:06:00Z)
• Sobolev Training for Implicit Neural Representations with Approximated Image Derivatives [12.71676484494428]
  Implicit Neural Representations (INRs) parameterized by neural networks have emerged as a powerful tool to represent different kinds of signals. We propose a training paradigm for INRs whose target output is image pixels, to encode image derivatives in addition to image values in the neural network. We show how the training paradigm can be leveraged to solve typical INRs problems, i.e., image regression and inverse rendering.
  arXiv  Detail & Related papers  (2022-07-21T10:12:41Z)
• Neural Implicit Dictionary via Mixture-of-Expert Training [111.08941206369508]
  We present a generic INR framework that achieves both data and training efficiency by learning a Neural Implicit Dictionary (NID) Our NID assembles a group of coordinate-based Impworks which are tuned to span the desired function space. Our experiments show that, NID can improve reconstruction of 2D images or 3D scenes by 2 orders of magnitude faster with up to 98% less input data.
  arXiv  Detail & Related papers  (2022-07-08T05:07:19Z)
• Reducing Redundancy in the Bottleneck Representation of the Autoencoders [98.78384185493624]
  Autoencoders are a type of unsupervised neural networks, which can be used to solve various tasks. We propose a scheme to explicitly penalize feature redundancies in the bottleneck representation. We tested our approach across different tasks: dimensionality reduction using three different dataset, image compression using the MNIST dataset, and image denoising using fashion MNIST.
  arXiv  Detail & Related papers  (2022-02-09T18:48:02Z)
• Meta-Learning Sparse Implicit Neural Representations [69.15490627853629]
  Implicit neural representations are a promising new avenue of representing general signals. Current approach is difficult to scale for a large number of signals or a data set. We show that meta-learned sparse neural representations achieve a much smaller loss than dense meta-learned models.
  arXiv  Detail & Related papers  (2021-10-27T18:02:53Z)
• Learning to Time-Decode in Spiking Neural Networks Through the Information Bottleneck [37.376989855065545]
  One of the key challenges in training Spiking Neural Networks (SNNs) is that target outputs typically come in the form of natural signals. This is done by handcrafting target spiking signals, which in turn implicitly fixes the mechanisms used to decode spikes into natural signals. This work introduces a hybrid variational autoencoder architecture, consisting of an encoding SNN and a decoding Artificial Neural Network.
  arXiv  Detail & Related papers  (2021-06-02T14:14:47Z)
• Deep Networks for Direction-of-Arrival Estimation in Low SNR [89.45026632977456]
  We introduce a Convolutional Neural Network (CNN) that is trained from mutli-channel data of the true array manifold matrix. We train a CNN in the low-SNR regime to predict DoAs across all SNRs. Our robust solution can be applied in several fields, ranging from wireless array sensors to acoustic microphones or sonars.
  arXiv  Detail & Related papers  (2020-11-17T12:52:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.