Adaptive Training of INRs via Pruning and Densification

• URL: http://arxiv.org/abs/2510.23943v1
• Date: Mon, 27 Oct 2025 23:52:46 GMT
• Title: Adaptive Training of INRs via Pruning and Densification
• Authors: Diana Aldana, João Paulo Lima, Daniel Csillag, Daniel Perazzo, Haoan Feng, Luiz Velho, Tiago Novello,
• Abstract summary: We introduce AIRe, an adaptive training scheme that refines the implicit neural representations over the course of optimization.<n>Our method uses a neuron pruning mechanism to avoid redundancy and input frequency densification to improve representation capacity.<n>Code and pretrained models will be released for public use.
• Score: 6.759337697337581
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Encoding input coordinates with sinusoidal functions into multilayer perceptrons (MLPs) has proven effective for implicit neural representations (INRs) of low-dimensional signals, enabling the modeling of high-frequency details. However, selecting appropriate input frequencies and architectures while managing parameter redundancy remains an open challenge, often addressed through heuristics and heavy hyperparameter optimization schemes. In this paper, we introduce AIRe ($\textbf{A}$daptive $\textbf{I}$mplicit neural $\textbf{Re}$presentation), an adaptive training scheme that refines the INR architecture over the course of optimization. Our method uses a neuron pruning mechanism to avoid redundancy and input frequency densification to improve representation capacity, leading to an improved trade-off between network size and reconstruction quality. For pruning, we first identify less-contributory neurons and apply a targeted weight decay to transfer their information to the remaining neurons, followed by structured pruning. Next, the densification stage adds input frequencies to spectrum regions where the signal underfits, expanding the representational basis. Through experiments on images and SDFs, we show that AIRe reduces model size while preserving, or even improving, reconstruction quality. Code and pretrained models will be released for public use.

Related papers

• Improving Accuracy and Efficiency of Implicit Neural Representations: Making SIREN a WINNER [0.0]
  We identify and address a fundamental limitation of sinusoidal representation networks (SIRENs)<n>In extreme cases, when the network's frequency support misaligns with the target spectrum, a'spectral bottleneck' is observed.<n>We propose WINNER - Weight Initialization with Noise for Neural Representations.
  arXiv  Detail & Related papers  (2025-09-16T11:41:13Z)
• SBS: Enhancing Parameter-Efficiency of Neural Representations for Neural Networks via Spectral Bias Suppression [6.410718573605]
  Implicit neural representations have been extended to represent convolutional neural network weights via neural representation for neural networks.<n>Standard multi-layer perceptrons used in neural representation for neural networks exhibit a pronounced spectral bias.<n>We propose SBS, a parameter-efficient enhancement to neural representation for neural networks that suppresses spectral bias.
  arXiv  Detail & Related papers  (2025-09-09T03:48:57Z)
• CS-SHRED: Enhancing SHRED for Robust Recovery of Spatiotemporal Dynamics [2.8820361301109365]
  We present CS-SHRED, a deep learning architecture that integrates CS into a Shallow Recurrent Decoder (SHRED) to reconstruct dynamics from incomplete, compressed, or corrupted data.<n>Compared to the traditional SHRED approach, CS-SHRED achieves significantly higher reconstruction fidelity -- as demonstrated by improved SSIM and PSNR values, lower normalized errors, and enhanced LPIPS scores.
  arXiv  Detail & Related papers  (2025-07-30T00:27:18Z)
• SING: Semantic Image Communications using Null-Space and INN-Guided Diffusion Models [52.40011613324083]
  Joint source-channel coding systems (DeepJSCC) have recently demonstrated remarkable performance in wireless image transmission.<n>Existing methods focus on minimizing distortion between the transmitted image and the reconstructed version at the receiver, often overlooking perceptual quality.<n>We propose SING, a novel framework that formulates the recovery of high-quality images from corrupted reconstructions as an inverse problem.
  arXiv  Detail & Related papers  (2025-03-16T12:32:11Z)
• Deep-Unrolling Multidimensional Harmonic Retrieval Algorithms on Neuromorphic Hardware [78.17783007774295]
  This paper explores the potential of conversion-based neuromorphic algorithms for highly accurate and energy-efficient single-snapshot multidimensional harmonic retrieval.<n>A novel method for converting the complex-valued convolutional layers and activations into spiking neural networks (SNNs) is developed.<n>The converted SNNs achieve almost five-fold power efficiency at moderate performance loss compared to the original CNNs.
  arXiv  Detail & Related papers  (2024-12-05T09:41:33Z)
• Tuning the Frequencies: Robust Training for Sinusoidal Neural Networks [1.5124439914522694]
  We introduce a theoretical framework that explains the capacity property of sinusoidal networks.<n>We show how its layer compositions produce a large number of new frequencies expressed as integer combinations of the input frequencies.<n>Our method, referred to as TUNER, greatly improves the stability and convergence of sinusoidal INR training, leading to detailed reconstructions.
  arXiv  Detail & Related papers  (2024-07-30T18:24:46Z)
• Spatial Annealing for Efficient Few-shot Neural Rendering [73.49548565633123]
  We introduce an accurate and efficient few-shot neural rendering method named textbfSpatial textbfAnnealing regularized textbfNeRF (textbfSANeRF)<n>By adding merely one line of code, SANeRF delivers superior rendering quality and much faster reconstruction speed compared to current few-shot neural rendering methods.
  arXiv  Detail & Related papers  (2024-06-12T02:48:52Z)
• INCODE: Implicit Neural Conditioning with Prior Knowledge Embeddings [4.639495398851869]
  Implicit Neural Representations (INRs) have revolutionized signal representation by leveraging neural networks to provide continuous and smooth representations of complex data. We introduce INCODE, a novel approach that enhances the control of the sinusoidal-based activation function in INRs using deep prior knowledge. Our approach not only excels in representation, but also extends its prowess to tackle complex tasks such as audio, image, and 3D shape reconstructions.
  arXiv  Detail & Related papers  (2023-10-28T23:16:49Z)
• NAF: Neural Attenuation Fields for Sparse-View CBCT Reconstruction [79.13750275141139]
  This paper proposes a novel and fast self-supervised solution for sparse-view CBCT reconstruction. The desired attenuation coefficients are represented as a continuous function of 3D spatial coordinates, parameterized by a fully-connected deep neural network. A learning-based encoder entailing hash coding is adopted to help the network capture high-frequency details.
  arXiv  Detail & Related papers  (2022-09-29T04:06:00Z)
• Deep Networks for Direction-of-Arrival Estimation in Low SNR [89.45026632977456]
  We introduce a Convolutional Neural Network (CNN) that is trained from mutli-channel data of the true array manifold matrix. We train a CNN in the low-SNR regime to predict DoAs across all SNRs. Our robust solution can be applied in several fields, ranging from wireless array sensors to acoustic microphones or sonars.
  arXiv  Detail & Related papers  (2020-11-17T12:52:18Z)
• Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networks [54.27962244835622]
  This paper proposes a new mean-field framework for over- parameterized deep neural networks (DNNs) In this framework, a DNN is represented by probability measures and functions over its features in the continuous limit. We illustrate the framework via the standard DNN and the Residual Network (Res-Net) architectures.
  arXiv  Detail & Related papers  (2020-07-03T01:37:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.