WeightCLIP: Aligning Datasets and Models for Weight Space Learning

• URL: http://arxiv.org/abs/2607.03551v1
• Date: Fri, 03 Jul 2026 18:27:18 GMT
• Title: WeightCLIP: Aligning Datasets and Models for Weight Space Learning
• Abstract summary: We propose WeightCLIP, a method for learning a dataset-aligned latent space for neural networks.<n>The NNs are encoded as latent representations using an autoencoder, while dataset samples are encoded using a dataset encoder.<n>We demonstrate that such representations can be used for different downstream tasks, including mapping dataset information to a weight-space representation that decodes to strong models.
• Score: 7.687988999964667
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Weight space learning aims to learn representations of neural network (NN) weights, enabling different downstream tasks. Existing approaches show promising performance, but lacking a way to shape these weight-space representations using information about the datasets the models were trained on, thus limiting downstream applications. We propose WeightCLIP, a method for learning a dataset-aligned latent space for neural networks, where datasets information is induced during training. The NNs are encoded as latent representations using an autoencoder, while dataset samples are encoded using a dataset encoder. The two representations are aligned using a contrastive objective, effectively reshaping the weight-space representations according to the datasets. We demonstrate that such representations can be used for different downstream tasks, including mapping dataset information to a weight-space representation that decode to strong models. In addition, we introduce a latent refinement process for generating models that outperforms standard fine-tuning. Overall, our results demonstrate that explicitly incorporating dataset information improves what can be achieved with weight-space representations across retrieval, generation, and refinement. Code will be available at https://github.com/HSG-AIML/WeightCLIP.

Related papers

• A Survey of Weight Space Learning: Understanding, Representation, and Generation [87.2501236345188]
  Weight Space Learning treats neural weights as a meaningful domain for analysis and modeling.<n>We categorize existing methods into three core dimensions: Weight Space Understanding, Weight Space Representation, and Weight Space Generation.<n>We show how these developments enable practical applications, including model retrieval, continual and federated learning, neural architecture search, and data-free reconstruction.
  arXiv  Detail & Related papers  (2026-03-10T15:32:23Z)
• Ensuring Semantics in Weights of Implicit Neural Representations through the Implicit Function Theorem [4.650983048077127]
  Implicit Neural Representations (INRs) provide a convenient testbed for Weight Space Learning (WSL)<n>We deploy the Implicit Function Theorem (IFT) to establish a rigorous mapping between the data space and its latent weight representation space.<n>We analyze a framework that maps instance-specific embeddings to INR weights via a shared hypernetwork.
  arXiv  Detail & Related papers  (2026-01-30T17:05:37Z)
• Bringing Balance to Hand Shape Classification: Mitigating Data Imbalance Through Generative Models [0.0]
  We use an EfficientNet classifier trained on the RWTH German sign language handshape dataset.<n>Our proposed techniques improve the current state-of-the-art accuracy on the RWTH dataset by 5%.
  arXiv  Detail & Related papers  (2025-07-22T20:41:29Z)
• Diffusion-Based Neural Network Weights Generation [80.89706112736353]
  D2NWG is a diffusion-based neural network weights generation technique that efficiently produces high-performing weights for transfer learning. Our method extends generative hyper-representation learning to recast the latent diffusion paradigm for neural network weights generation. Our approach is scalable to large architectures such as large language models (LLMs), overcoming the limitations of current parameter generation techniques.
  arXiv  Detail & Related papers  (2024-02-28T08:34:23Z)
• Learn to Unlearn for Deep Neural Networks: Minimizing Unlearning Interference with Gradient Projection [56.292071534857946]
  Recent data-privacy laws have sparked interest in machine unlearning. Challenge is to discard information about the forget'' data without altering knowledge about remaining dataset. We adopt a projected-gradient based learning method, named as Projected-Gradient Unlearning (PGU) We provide empirically evidence to demonstrate that our unlearning method can produce models that behave similar to models retrained from scratch across various metrics even when the training dataset is no longer accessible.
  arXiv  Detail & Related papers  (2023-12-07T07:17:24Z)
• Data Augmentations in Deep Weight Spaces [89.45272760013928]
  We introduce a novel augmentation scheme based on the Mixup method. We evaluate the performance of these techniques on existing benchmarks as well as new benchmarks we generate.
  arXiv  Detail & Related papers  (2023-11-15T10:43:13Z)
• Dataset Quantization [72.61936019738076]
  We present dataset quantization (DQ), a new framework to compress large-scale datasets into small subsets. DQ is the first method that can successfully distill large-scale datasets such as ImageNet-1k with a state-of-the-art compression ratio.
  arXiv  Detail & Related papers  (2023-08-21T07:24:29Z)
• DatasetDM: Synthesizing Data with Perception Annotations Using Diffusion Models [61.906934570771256]
  We present a generic dataset generation model that can produce diverse synthetic images and perception annotations. Our method builds upon the pre-trained diffusion model and extends text-guided image synthesis to perception data generation. We show that the rich latent code of the diffusion model can be effectively decoded as accurate perception annotations using a decoder module.
  arXiv  Detail & Related papers  (2023-08-11T14:38:11Z)
• Learning a Self-Expressive Network for Subspace Clustering [15.096251922264281]
  We propose a novel framework for subspace clustering, termed Self-Expressive Network (SENet), which employs a properly designed neural network to learn a self-expressive representation of the data. Our SENet can not only learn the self-expressive coefficients with desired properties on the training data, but also handle out-of-sample data. In particular, SENet yields highly competitive performance on MNIST, Fashion MNIST and Extended MNIST and state-of-the-art performance on CIFAR-10.
  arXiv  Detail & Related papers  (2021-10-08T18:06:06Z)
• DeepSatData: Building large scale datasets of satellite images for training machine learning models [77.17638664503215]
  This report presents design considerations for automatically generating satellite imagery datasets for training machine learning models. We discuss issues faced from the point of view of deep neural network training and evaluation.
  arXiv  Detail & Related papers  (2021-04-28T15:13:12Z)
• Classifying the classifier: dissecting the weight space of neural networks [16.94879659770577]
  This paper presents an empirical study on the weights of neural networks. We interpret each model as a point in a high-dimensional space -- the neural weight space. To promote further research on the weight space, we release the neural weight space (NWS) dataset.
  arXiv  Detail & Related papers  (2020-02-13T18:12:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.