Ensuring Semantics in Weights of Implicit Neural Representations through the Implicit Function Theorem

• URL: http://arxiv.org/abs/2601.23181v1
• Date: Fri, 30 Jan 2026 17:05:37 GMT
• Title: Ensuring Semantics in Weights of Implicit Neural Representations through the Implicit Function Theorem
• Abstract summary: Implicit Neural Representations (INRs) provide a convenient testbed for Weight Space Learning (WSL)<n>We deploy the Implicit Function Theorem (IFT) to establish a rigorous mapping between the data space and its latent weight representation space.<n>We analyze a framework that maps instance-specific embeddings to INR weights via a shared hypernetwork.
• Score: 4.650983048077127
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Weight Space Learning (WSL), which frames neural network weights as a data modality, is an emerging field with potential for tasks like meta-learning or transfer learning. Particularly, Implicit Neural Representations (INRs) provide a convenient testbed, where each set of weights determines the corresponding individual data sample as a mapping from coordinates to contextual values. So far, a precise theoretical explanation for the mechanism of encoding semantics of data into network weights is still missing. In this work, we deploy the Implicit Function Theorem (IFT) to establish a rigorous mapping between the data space and its latent weight representation space. We analyze a framework that maps instance-specific embeddings to INR weights via a shared hypernetwork, achieving performance competitive with existing baselines on downstream classification tasks across 2D and 3D datasets. These findings offer a theoretical lens for future investigations into network weights.

Related papers

• Weight-Space Mixture-of-Experts for Implicit Neural Representation Classification [0.20625936401496228]
  Implicit Neural Representations (INRs) encode signals as the weights of a coordinate-based neural network.<n>While promising, classification directly in weight space remains challenging due to the high dimensionality and complex structure of INR parameters.<n>We propose a hierarchical Mixture-of-Experts (HMoE) Transformer that processes INR weights using conditional computation aligned with the structure of the underlying network.
  arXiv  Detail & Related papers  (2026-07-31T14:28:33Z)
• WeightCLIP: Aligning Datasets and Models for Weight Space Learning [7.687988999964667]
  We propose WeightCLIP, a method for learning a dataset-aligned latent space for neural networks.<n>The NNs are encoded as latent representations using an autoencoder, while dataset samples are encoded using a dataset encoder.<n>We demonstrate that such representations can be used for different downstream tasks, including mapping dataset information to a weight-space representation that decodes to strong models.
  arXiv  Detail & Related papers  (2026-07-03T18:27:18Z)
• A Survey of Weight Space Learning: Understanding, Representation, and Generation [87.2501236345188]
  Weight Space Learning treats neural weights as a meaningful domain for analysis and modeling.<n>We categorize existing methods into three core dimensions: Weight Space Understanding, Weight Space Representation, and Weight Space Generation.<n>We show how these developments enable practical applications, including model retrieval, continual and federated learning, neural architecture search, and data-free reconstruction.
  arXiv  Detail & Related papers  (2026-03-10T15:32:23Z)
• Interpretability of the Intent Detection Problem: A New Approach [0.0]
  Internal mechanisms enabling Recurrent Neural Networks to solve intent detection tasks are poorly understood.<n>We apply dynamical systems theory to analyze how RNN architectures address this problem.<n>Our framework decouples geometric separation from readout alignment, providing a novel, mechanistic explanation for real world performance disparities.
  arXiv  Detail & Related papers  (2026-01-23T20:27:47Z)
• Unveiling the Training Dynamics of ReLU Networks through a Linear Lens [0.0]
  We recast a multi-layer ReLU network into a single-layer linear model with input-dependent "effective weights"<n>For any given input sample, the activation pattern of ReLU units creates a unique computational path.<n>As training progresses, the effective weights corresponding to samples from the same class converge, while those from different classes diverge.
  arXiv  Detail & Related papers  (2025-11-07T02:59:51Z)
• Knowledge-Informed Neural Network for Complex-Valued SAR Image Recognition [51.03674130115878]
  We introduce the Knowledge-Informed Neural Network (KINN), a lightweight framework built upon a novel "compression-aggregation-compression" architecture.<n>KINN establishes a state-of-the-art in parameter-efficient recognition, offering exceptional generalization in data-scarce and out-of-distribution scenarios.
  arXiv  Detail & Related papers  (2025-10-23T07:12:26Z)
• Data Augmentations in Deep Weight Spaces [89.45272760013928]
  We introduce a novel augmentation scheme based on the Mixup method. We evaluate the performance of these techniques on existing benchmarks as well as new benchmarks we generate.
  arXiv  Detail & Related papers  (2023-11-15T10:43:13Z)
• Functional Indirection Neural Estimator for Better Out-of-distribution Generalization [27.291114360472243]
  FINE (Functional Indirection Neural Estorimator) learns to compose functions that map data input to output on-the-fly. We train FINE and competing models on IQ tasks using images from the MNIST, Omniglot and CIFAR100 datasets. FINE not only achieves the best performance on all tasks but also is able to adapt to small-scale data scenarios.
  arXiv  Detail & Related papers  (2022-10-23T14:43:02Z)
• Decomposing neural networks as mappings of correlation functions [57.52754806616669]
  We study the mapping between probability distributions implemented by a deep feed-forward network. We identify essential statistics in the data, as well as different information representations that can be used by neural networks.
  arXiv  Detail & Related papers  (2022-02-10T09:30:31Z)
• Locally Sparse Networks for Interpretable Predictions [7.362415721170984]
  We propose a framework for training locally sparse neural networks where the local sparsity is learned via a sample-specific gating mechanism. The sample-specific sparsity is predicted via a textitgating network, which is trained in tandem with the textitprediction network. We demonstrate that our method outperforms state-of-the-art models when predicting the target function with far fewer features per instance.
  arXiv  Detail & Related papers  (2021-06-11T15:46:50Z)
• PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning [109.84770951839289]
  We present PredRNN, a new recurrent network for learning visual dynamics from historical context. We show that our approach obtains highly competitive results on three standard datasets.
  arXiv  Detail & Related papers  (2021-03-17T08:28:30Z)
• Estimating informativeness of samples with Smooth Unique Information [108.25192785062367]
  We measure how much a sample informs the final weights and how much it informs the function computed by the weights. We give efficient approximations of these quantities using a linearized network. We apply these measures to several problems, such as dataset summarization.
  arXiv  Detail & Related papers  (2021-01-17T10:29:29Z)
• Neural networks adapting to datasets: learning network size and topology [77.34726150561087]
  We introduce a flexible setup allowing for a neural network to learn both its size and topology during the course of a gradient-based training. The resulting network has the structure of a graph tailored to the particular learning task and dataset.
  arXiv  Detail & Related papers  (2020-06-22T12:46:44Z)
• Hierarchical Gaussian Process Priors for Bayesian Neural Network Weights [16.538973310830414]
  A desirable class of priors would represent weights compactly, capture correlations between weights, and allow inclusion of prior knowledge. This paper introduces two innovations: (i) a process-based hierarchical model for network weights based on unit embeddings that can flexibly encode correlated weight structures, and (ii) input-dependent versions of these weight priors that can provide convenient ways to regularize the function space. We show these models provide desirable test-time uncertainty estimates on out-of-distribution data, demonstrate cases of modeling inductive biases for neural networks with kernels, and demonstrate competitive predictive performance on an active learning benchmark
  arXiv  Detail & Related papers  (2020-02-10T07:19:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.