FiRe: Frequency Reparameterization as a Preconditioner for Periodic Implicit Neural Representations

• URL: http://arxiv.org/abs/2606.29414v1
• Date: Sun, 28 Jun 2026 14:24:51 GMT
• Title: FiRe: Frequency Reparameterization as a Preconditioner for Periodic Implicit Neural Representations
• Abstract summary: We introduce FiRe, that accelerates optimization by re parameterizing per-neuron frequency of periodic INRs without changing their activation function.<n>We characterize how performance depends on resolution, rank, and training budget, and give an NTK account that predicts these trends.
• Score: 0.09999629695552194
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Periodic Implicit Neural Representations (INRs) such as SIREN and FINER assign every neuron, the same global frequency, spending the representational budget inefficiently when local signal content varies. We introduce FiRe (Frequency Reparameterization), that accelerates optimization by reparameterizing per-neuron frequency of periodic INRs without changing their underlying activation function. FiRe gives each neuron a bounded, input-dependent frequency via a separate low-rank gating path and is applicable to any periodic activation function. The gate acts as an implicit preconditioner that improves optimization conditioning at initialization via the Neural Tangent Kernel (NTK). This better-conditioned initialization makes optimization converge faster, and the high-frequency content of the reconstruction tracks the target more closely at a fixed computational budget. On 2D image fitting, FiRe increases PSNR over a parameter-matched baseline (up to +1 dB at short training budgets), with gains that vary with resolution and diminish at full convergence. We characterize how performance depends on resolution, rank, and training budget, and give an NTK account that predicts these trends.

Related papers

• Spectral Gating via Damped Oscillations for Adaptive Implicit Neural Representations [24.706105488382818]
  Implicit Neural Representations (INRs) have been proven successful in encoding continuous signals through coordinate-based networks.<n>They face a spectral dilemma: periodic activations capture fine details but act as all-pass filters that memorise noise, while spatially compact activations regularise effectively but suffer from low-frequency bias.<n>We propose to model each neuron's activation as the steady-state response of a sinusoidally-forced damped harmonic oscillator.
  arXiv  Detail & Related papers  (2026-06-22T11:59:47Z)
• Deciphering Neural Reparameterized Full-Waveform Inversion with Neural Sensitivity Kernel and Wave Tangent Kernel [67.14508461124932]
  Full-waveform inversion (FWI) estimates unknown parameters in the wave equation from limited boundary measurements.<n>Recent advances in neural reparameterized FWI (NeurFWI) demonstrate that representing the parameters using a neural network can reduce the reliance on the high-quality initial model and wavefield data.<n>In this study, we establish the neural sensitivity kernel (NSK) and the wave tangent kernel (WTK) to analyze their convergence behavior from both model and data domains.
  arXiv  Detail & Related papers  (2026-05-14T04:49:23Z)
• Unveiling the Mechanism of Continuous Representation Full-Waveform Inversion: A Wave Based Neural Tangent Kernel Framework [52.632217600064344]
  Full-waveform inversion (FWI) estimates physical parameters in the wave equation from limited measurements.<n>FWI methods are limited by their notorious sensitivity to the accuracy of the initial models.<n>Recent progress in continuous representation FWI (CR-FWI) demonstrates that representing parameter models with a coordinate-based neural network, such as implicit neural representation (INR), can mitigate the dependence on initial models.
  arXiv  Detail & Related papers  (2026-03-23T03:22:33Z)
• FUTON: Fourier Tensor Network for Implicit Neural Representations [56.48739018255443]
  Implicit neural representations (INRs) have emerged as powerful tools for encoding signals, yet dominant-based designs often suffer from slow convergence, overfitting to noise, and poor extrapolation.<n>We introduce FUTON, which models signals as generalized Fourier series whose coefficients are parameterized by a low-rank tensor decomposition.
  arXiv  Detail & Related papers  (2026-02-13T19:31:44Z)
• Adaptive Training of INRs via Pruning and Densification [6.759337697337581]
  We introduce AIRe, an adaptive training scheme that refines the implicit neural representations over the course of optimization.<n>Our method uses a neuron pruning mechanism to avoid redundancy and input frequency densification to improve representation capacity.<n>Code and pretrained models will be released for public use.
  arXiv  Detail & Related papers  (2025-10-27T23:52:46Z)
• FM-SIREN & FM-FINER: Nyquist-Informed Frequency Multiplier for Implicit Neural Representation with Periodic Activation [15.456760941404873]
  We propose FM-SIREN and FM-FINER, which assign Nyquist-informed, neuron-specific frequency multipliers to periodic activations.<n>This simple yet principled modification reduces the redundancy of features by nearly 50% and consistently improves signal reconstruction across diverse INR tasks.
  arXiv  Detail & Related papers  (2025-09-27T18:14:47Z)
• Neuromorphic Wireless Split Computing with Resonate-and-Fire Neurons [69.73249913506042]
  This paper investigates a wireless split computing architecture that employs resonate-and-fire (RF) neurons to process time-domain signals directly.<n>By resonating at tunable frequencies, RF neurons extract time-localized spectral features while maintaining low spiking activity.<n> Experimental results show that the proposed RF-SNN architecture achieves comparable accuracy to conventional LIF-SNNs and ANNs.
  arXiv  Detail & Related papers  (2025-06-24T21:14:59Z)
• Cross-Frequency Implicit Neural Representation with Self-Evolving Parameters [52.574661274784916]
  Implicit neural representation (INR) has emerged as a powerful paradigm for visual data representation.<n>We propose a self-evolving cross-frequency INR using the Haar wavelet transform (termed CF-INR), which decouples data into four frequency components and employs INRs in the wavelet space.<n>We evaluate CF-INR on a variety of visual data representation and recovery tasks, including image regression, inpainting, denoising, and cloud removal.
  arXiv  Detail & Related papers  (2025-04-15T07:14:35Z)
• Tuning the Frequencies: Robust Training for Sinusoidal Neural Networks [1.5124439914522694]
  We introduce a theoretical framework that explains the capacity property of sinusoidal networks.<n>We show how its layer compositions produce a large number of new frequencies expressed as integer combinations of the input frequencies.<n>Our method, referred to as TUNER, greatly improves the stability and convergence of sinusoidal INR training, leading to detailed reconstructions.
  arXiv  Detail & Related papers  (2024-07-30T18:24:46Z)
• FINER++: Building a Family of Variable-periodic Functions for Activating Implicit Neural Representation [39.116375158815515]
  Implicit Neural Representation (INR) is causing a revolution in the field of signal processing. INR techniques suffer from the "frequency"-specified spectral bias and capacity-convergence gap. We propose the FINER++ framework by extending existing periodic/non-periodic activation functions to variable-periodic ones.
  arXiv  Detail & Related papers  (2024-07-28T09:24:57Z)
• Spatial Annealing for Efficient Few-shot Neural Rendering [73.49548565633123]
  We introduce an accurate and efficient few-shot neural rendering method named textbfSpatial textbfAnnealing regularized textbfNeRF (textbfSANeRF)<n>By adding merely one line of code, SANeRF delivers superior rendering quality and much faster reconstruction speed compared to current few-shot neural rendering methods.
  arXiv  Detail & Related papers  (2024-06-12T02:48:52Z)
• Learning Frequency Domain Approximation for Binary Neural Networks [68.79904499480025]
  We propose to estimate the gradient of sign function in the Fourier frequency domain using the combination of sine functions for training BNNs. The experiments on several benchmark datasets and neural architectures illustrate that the binary network learned using our method achieves the state-of-the-art accuracy.
  arXiv  Detail & Related papers  (2021-03-01T08:25:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.