Weight Space Representation Learning with Neural Fields

• URL: http://arxiv.org/abs/2512.01759v1
• Date: Mon, 01 Dec 2025 15:05:01 GMT
• Title: Weight Space Representation Learning with Neural Fields
• Authors: Zhuoqian Yang, Mathieu Salzmann, Sabine Süsstrunk,
• Abstract summary: In this work, we investigate the potential of weights to serve as effective representations, focusing on neural fields.<n>Our key insight is that constraining the optimization space through a pre-trained base model and low-rank adaptation (LoRA) can induce structure in weight space.<n>Across reconstruction, generation, and analysis tasks on 2D and 3D data, we find that multiplicative LoRA weights achieve high representation quality while exhibiting distinctiveness and semantic structure.
• Score: 69.85677017502826
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In this work, we investigate the potential of weights to serve as effective representations, focusing on neural fields. Our key insight is that constraining the optimization space through a pre-trained base model and low-rank adaptation (LoRA) can induce structure in weight space. Across reconstruction, generation, and analysis tasks on 2D and 3D data, we find that multiplicative LoRA weights achieve high representation quality while exhibiting distinctiveness and semantic structure. When used with latent diffusion models, multiplicative LoRA weights enable higher-quality generation than existing weight-space methods.

Related papers

• COOPER: A Unified Model for Cooperative Perception and Reasoning in Spatial Intelligence [57.63155257058967]
  We investigate whether a unified MLLM can develop an intrinsic ability to enhance spatial perception and, through adaptive interleaved reasoning, achieve stronger spatial intelligence.<n>We propose textbfCOOPER, a unified MLLM that leverages depth and segmentation as auxiliary modalities and is trained in two stages to acquire auxiliary modality generation and adaptive, interleaved reasoning capabilities.
  arXiv  Detail & Related papers  (2025-12-04T08:26:04Z)
• Geometric Flow Models over Neural Network Weights [0.0]
  A generative model of neural network weights would be useful for a diverse set of applications, such as deep learning, learned optimization, and transfer learning.<n>Existing work on weight-space generative models often ignores the symmetries of neural network weights, or only takes into account a subset of them.<n>We build on recent work on generative modeling with flow matching, and weight-space graph neural networks to design three different weight-space flows.
  arXiv  Detail & Related papers  (2025-03-27T19:29:44Z)
• Shape Generation via Weight Space Learning [12.429026910048528]
  We show that submanifolds within a large 3D shape-generative model can modulate topological properties or fine-grained part features separately.<n>Results highlight the potential of weight space learning to unlock new approaches for 3D shape generation and specialized fine-tuning.
  arXiv  Detail & Related papers  (2025-03-26T15:49:27Z)
• Structure Is Not Enough: Leveraging Behavior for Neural Network Weight Reconstruction [6.926413609535758]
  One approach to leverage NN weights involves training autoencoders (AEs) using contrastive and reconstruction losses.<n>AEs reconstruct NN models with deteriorated performance compared to the original ones, limiting their usability with regard to model weight generation.<n>We show a strong synergy between structural and behavioral signals, leading to increased performance in all downstream tasks evaluated.
  arXiv  Detail & Related papers  (2025-03-21T13:39:04Z)
• LoRA vs Full Fine-tuning: An Illusion of Equivalence [73.5303340531806]
  We study how Low-Rank Adaptation (LoRA) and full-finetuning change pre-trained models.<n>We find that LoRA and full fine-tuning yield weight matrices whose singular value decompositions exhibit very different structure.<n>We extend the finding that LoRA forgets less than full fine-tuning and find its forgetting is vastly localized to the intruder dimension.
  arXiv  Detail & Related papers  (2024-10-28T17:14:01Z)
• Learning on LoRAs: GL-Equivariant Processing of Low-Rank Weight Spaces for Large Finetuned Models [38.197552424549514]
  Low-rank adaptations (LoRAs) have revolutionized the finetuning of large foundation models. LoRAs present opportunities for applying machine learning techniques that take these low-rank weights themselves as inputs. In this paper, we investigate the potential of Learning on LoRAs (LoL), a paradigm where LoRA weights serve as input to machine learning models.
  arXiv  Detail & Related papers  (2024-10-05T15:52:47Z)
• DiffLoRA: Generating Personalized Low-Rank Adaptation Weights with Diffusion [43.55179971287028]
  We propose DiffLoRA, an efficient method that leverages the diffusion model as a hypernetwork to predict personalized Low-Rank Adaptation weights. By incorporating these LoRA weights into the off-the-shelf text-to-image model, DiffLoRA enables zero-shot personalization during inference. We introduce a novel identity-oriented LoRA weights construction pipeline to facilitate the training process of DiffLoRA.
  arXiv  Detail & Related papers  (2024-08-13T09:00:35Z)
• Diffusion-Based Neural Network Weights Generation [80.89706112736353]
  D2NWG is a diffusion-based neural network weights generation technique that efficiently produces high-performing weights for transfer learning. Our method extends generative hyper-representation learning to recast the latent diffusion paradigm for neural network weights generation. Our approach is scalable to large architectures such as large language models (LLMs), overcoming the limitations of current parameter generation techniques.
  arXiv  Detail & Related papers  (2024-02-28T08:34:23Z)
• Improved Generalization of Weight Space Networks via Augmentations [53.87011906358727]
  Learning in deep weight spaces (DWS) is an emerging research direction, with applications to 2D and 3D neural fields (INRs, NeRFs) We empirically analyze the reasons for this overfitting and find that a key reason is the lack of diversity in DWS datasets. To address this, we explore strategies for data augmentation in weight spaces and propose a MixUp method adapted for weight spaces.
  arXiv  Detail & Related papers  (2024-02-06T15:34:44Z)
• Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Models [96.9373147383119]
  We show that weight disentanglement is the crucial factor that makes task arithmetic effective. We show that fine-tuning models in their tangent space by linearizing them amplifies weight disentanglement. This leads to substantial performance improvements across task arithmetic benchmarks and diverse models.
  arXiv  Detail & Related papers  (2023-05-22T08:39:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.