Feature Unlearning for Pre-trained GANs and VAEs

• URL: http://arxiv.org/abs/2303.05699v4
• Date: Thu, 28 Mar 2024 03:48:40 GMT
• Title: Feature Unlearning for Pre-trained GANs and VAEs
• Authors: Saemi Moon, Seunghyuk Cho, Dongwoo Kim,
• Abstract summary: We tackle the problem of feature unlearning from a pre-trained image generative model: GANs and VAEs. We aim to unlearn a specific feature, such as hairstyle from facial images, from the pre-trained generative models.
• Score: 4.80598572788967
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We tackle the problem of feature unlearning from a pre-trained image generative model: GANs and VAEs. Unlike a common unlearning task where an unlearning target is a subset of the training set, we aim to unlearn a specific feature, such as hairstyle from facial images, from the pre-trained generative models. As the target feature is only presented in a local region of an image, unlearning the entire image from the pre-trained model may result in losing other details in the remaining region of the image. To specify which features to unlearn, we collect randomly generated images that contain the target features. We then identify a latent representation corresponding to the target feature and then use the representation to fine-tune the pre-trained model. Through experiments on MNIST, CelebA, and FFHQ datasets, we show that target features are successfully removed while keeping the fidelity of the original models. Further experiments with an adversarial attack show that the unlearned model is more robust under the presence of malicious parties.

Related papers

• Data Attribution for Text-to-Image Models by Unlearning Synthesized Images [71.23012718682634]
  The goal of data attribution for text-to-image models is to identify the training images that most influence the generation of a new image. We propose a new approach that efficiently identifies highly-influential images.
  arXiv  Detail & Related papers  (2024-06-13T17:59:44Z)
• Learning Embeddings with Centroid Triplet Loss for Object Identification in Robotic Grasping [14.958823096408175]
  Foundation models are a strong trend in deep learning and computer vision. Here, we focus on training such an object identification model. Key solution to train such a model is the centroid triplet loss (CTL), which aggregates image features to their centroids.
  arXiv  Detail & Related papers  (2024-04-09T13:01:26Z)
• DreamTeacher: Pretraining Image Backbones with Deep Generative Models [103.62397699392346]
  We introduce a self-supervised feature representation learning framework that utilizes generative networks for pre-training downstream image backbones. We investigate two types of knowledge distillation: 1) distilling learned generative features onto target image backbones as an alternative to pretraining these backbones on large labeled datasets such as ImageNet. We empirically find that our DreamTeacher significantly outperforms existing self-supervised representation learning approaches across the board.
  arXiv  Detail & Related papers  (2023-07-14T17:17:17Z)
• Supervised Deep Learning for Content-Aware Image Retargeting with Fourier Convolutions [11.031841470875571]
  Image aims to alter the size of the image with attention to the contents. Labeled datasets are unavailable for training deep learning models in the image tasks. Regular convolutional neural networks cannot generate images of different sizes in inference time.
  arXiv  Detail & Related papers  (2023-06-12T19:17:44Z)
• Exploring Target Representations for Masked Autoencoders [78.57196600585462]
  We show that a careful choice of the target representation is unnecessary for learning good representations. We propose a multi-stage masked distillation pipeline and use a randomly model as the teacher. A proposed method to perform masked knowledge distillation with bootstrapped teachers (dBOT) outperforms previous self-supervised methods by nontrivial margins.
  arXiv  Detail & Related papers  (2022-09-08T16:55:19Z)
• Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervision [38.22842778742829]
  Discriminative self-supervised learning allows training models on any random group of internet images. We train models on billions of random images without any data pre-processing or prior assumptions about what we want the model to learn. We extensively study and validate our model performance on over 50 benchmarks including fairness, to distribution shift, geographical diversity, fine grained recognition, image copy detection and many image classification datasets.
  arXiv  Detail & Related papers  (2022-02-16T22:26:47Z)
• Semantic-Aware Generation for Self-Supervised Visual Representation Learning [116.5814634936371]
  We advocate for Semantic-aware Generation (SaGe) to facilitate richer semantics rather than details to be preserved in the generated image. SaGe complements the target network with view-specific features and thus alleviates the semantic degradation brought by intensive data augmentations. We execute SaGe on ImageNet-1K and evaluate the pre-trained models on five downstream tasks including nearest neighbor test, linear classification, and fine-scaled image recognition.
  arXiv  Detail & Related papers  (2021-11-25T16:46:13Z)
• Meta Internal Learning [88.68276505511922]
  Internal learning for single-image generation is a framework, where a generator is trained to produce novel images based on a single image. We propose a meta-learning approach that enables training over a collection of images, in order to model the internal statistics of the sample image more effectively. Our results show that the models obtained are as suitable as single-image GANs for many common image applications.
  arXiv  Detail & Related papers  (2021-10-06T16:27:38Z)
• Rectifying the Shortcut Learning of Background: Shared Object Concentration for Few-Shot Image Recognition [101.59989523028264]
  Few-Shot image classification aims to utilize pretrained knowledge learned from a large-scale dataset to tackle a series of downstream classification tasks. We propose COSOC, a novel Few-Shot Learning framework, to automatically figure out foreground objects at both pretraining and evaluation stage.
  arXiv  Detail & Related papers  (2021-07-16T07:46:41Z)
• Distilling Visual Priors from Self-Supervised Learning [24.79633121345066]
  Convolutional Neural Networks (CNNs) are prone to overfit small training datasets. We present a novel two-phase pipeline that leverages self-supervised learning and knowledge distillation to improve the generalization ability of CNN models for image classification under the data-deficient setting.
  arXiv  Detail & Related papers  (2020-08-01T13:07:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.