Weight-Space Mixture-of-Experts for Implicit Neural Representation Classification

• URL: http://arxiv.org/abs/2607.29463v1
• Date: Fri, 31 Jul 2026 14:28:33 GMT
• Title: Weight-Space Mixture-of-Experts for Implicit Neural Representation Classification
• Abstract summary: Implicit Neural Representations (INRs) encode signals as the weights of a coordinate-based neural network.<n>While promising, classification directly in weight space remains challenging due to the high dimensionality and complex structure of INR parameters.<n>We propose a hierarchical Mixture-of-Experts (HMoE) Transformer that processes INR weights using conditional computation aligned with the structure of the underlying network.
• Score: 0.20625936401496228
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Implicit Neural Representations (INRs) encode signals as the weights of a coordinate-based neural network and have recently been proposed as an alternative domain for downstream learning. While promising, classification directly in weight space remains challenging due to the high dimensionality and complex structure of INR parameters. Furthermore, the way discriminative information is distributed across INR weights remains poorly understood. We propose a hierarchical Mixture-of-Experts (HMoE) Transformer that processes INR weights using conditional computation aligned with the structure of the underlying implicit network. Coupled with a meta-learning framework that shapes INR parameters for downstream tasks, our model achieves state-of-the-art accuracy across standard benchmarks, ranging from low-resolution datasets to high-resolution ImageNet-1K. To gain insight into how INRs encode discriminative information, we develop weight-space attribution and pruning methods that identify parameters most relevant for classification. These analyses reveal how class-specific structure emerges within INR layers and support the suitability of MoE architectures for weight-space learning. Our approach advances both the performance and interpretability of weight-space classifiers.

Related papers

• Compositional Sparsity as an Inductive Bias for Neural Architecture Design [35.06894725394093]
  We identify the structural priors that enable Deep Neural Networks to overcome the curse of dimensionality.<n>We present an interpretable pipeline in which abstraction emerges through hierarchical composition.<n>Across a broad suite of real-world datasets, HNNs consistently match or outperform dense baselines while using far fewer parameters.
  arXiv  Detail & Related papers  (2026-05-14T12:26:50Z)
• Ensuring Semantics in Weights of Implicit Neural Representations through the Implicit Function Theorem [4.650983048077127]
  Implicit Neural Representations (INRs) provide a convenient testbed for Weight Space Learning (WSL)<n>We deploy the Implicit Function Theorem (IFT) to establish a rigorous mapping between the data space and its latent weight representation space.<n>We analyze a framework that maps instance-specific embeddings to INR weights via a shared hypernetwork.
  arXiv  Detail & Related papers  (2026-01-30T17:05:37Z)
• Interpretability of the Intent Detection Problem: A New Approach [0.0]
  Internal mechanisms enabling Recurrent Neural Networks to solve intent detection tasks are poorly understood.<n>We apply dynamical systems theory to analyze how RNN architectures address this problem.<n>Our framework decouples geometric separation from readout alignment, providing a novel, mechanistic explanation for real world performance disparities.
  arXiv  Detail & Related papers  (2026-01-23T20:27:47Z)
• Knowledge-Informed Neural Network for Complex-Valued SAR Image Recognition [51.03674130115878]
  We introduce the Knowledge-Informed Neural Network (KINN), a lightweight framework built upon a novel "compression-aggregation-compression" architecture.<n>KINN establishes a state-of-the-art in parameter-efficient recognition, offering exceptional generalization in data-scarce and out-of-distribution scenarios.
  arXiv  Detail & Related papers  (2025-10-23T07:12:26Z)
• Detail Across Scales: Multi-Scale Enhancement for Full Spectrum Neural Representations [4.899720537787801]
  Implicit neural representations (INRs) have emerged as a compact and parametric alternative to discrete array-based data representations.<n>We propose WIEN-INR, a wavelet-informed implicit neural representation that distributes modeling across different resolution scales.<n>We show that WIEN-INR achieves superior reconstruction fidelity while maintaining a compact model size.
  arXiv  Detail & Related papers  (2025-09-19T00:15:39Z)
• SINR: Sparsity Driven Compressed Implicit Neural Representations [25.489983863030556]
  Implicit Neural Representations (INRs) are increasingly recognized as a versatile data modality for representing discretized signals.<n>Existing signal compression approaches for INRs typically employ one of two strategies.<n>We introduce SINR, an innovative compression algorithm that leverages the patterns in the vector spaces formed by weights of INRs.
  arXiv  Detail & Related papers  (2025-03-25T11:53:51Z)
• End-to-End Implicit Neural Representations for Classification [57.55927378696826]
  Implicit neural representations (INRs) encode a signal in neural network parameters and show excellent results for signal reconstruction.<n>INR-based classification still significantly under-performs compared to pixel-based methods like CNNs.<n>This work presents an end-to-end strategy for initializing SIRENs together with a learned learning-rate scheme.
  arXiv  Detail & Related papers  (2025-03-23T16:02:23Z)
• ARC: Anchored Representation Clouds for High-Resolution INR Classification [49.11170948406405]
  Implicit neural representations encode signals in neural network weights as memory-efficient representation.<n>Current INR image classification methods are sensitive to image-space transformations.<n>We propose ARC: Anchored Representation Clouds, a novel INR architecture that explicitly anchors latent vectors locally in image-space.
  arXiv  Detail & Related papers  (2025-03-19T12:24:29Z)
• Set-based Neural Network Encoding Without Weight Tying [91.37161634310819]
  We propose a neural network weight encoding method for network property prediction.<n>Our approach is capable of encoding neural networks in a model zoo of mixed architecture.<n>We introduce two new tasks for neural network property prediction: cross-dataset and cross-architecture.
  arXiv  Detail & Related papers  (2023-05-26T04:34:28Z)
• Signal Processing for Implicit Neural Representations [80.38097216996164]
  Implicit Neural Representations (INRs) encode continuous multi-media data via multi-layer perceptrons. Existing works manipulate such continuous representations via processing on their discretized instance. We propose an implicit neural signal processing network, dubbed INSP-Net, via differential operators on INR.
  arXiv  Detail & Related papers  (2022-10-17T06:29:07Z)
• Robust lEarned Shrinkage-Thresholding (REST): Robust unrolling for sparse recover [87.28082715343896]
  We consider deep neural networks for solving inverse problems that are robust to forward model mis-specifications. We design a new robust deep neural network architecture by applying algorithm unfolding techniques to a robust version of the underlying recovery problem. The proposed REST network is shown to outperform state-of-the-art model-based and data-driven algorithms in both compressive sensing and radar imaging problems.
  arXiv  Detail & Related papers  (2021-10-20T06:15:45Z)
• A Fully Tensorized Recurrent Neural Network [48.50376453324581]
  We introduce a "fully tensorized" RNN architecture which jointly encodes the separate weight matrices within each recurrent cell. This approach reduces model size by several orders of magnitude, while still maintaining similar or better performance compared to standard RNNs.
  arXiv  Detail & Related papers  (2020-10-08T18:24:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.