Model Stitching and Visualization How GAN Generators can Invert Networks in Real-Time

• URL: http://arxiv.org/abs/2302.02181v2
• Date: Tue, 20 Feb 2024 13:47:56 GMT
• Title: Model Stitching and Visualization How GAN Generators can Invert Networks in Real-Time
• Authors: Rudolf Herdt (1 and 2), Maximilian Schmidt (1 and 2), Daniel Otero Baguer (1 and 2), Jean Le'Clerc Arrastia (1 and 2), Peter Maass (1 and 2) ((1) University of Bremen, (2) aisencia)
• Abstract summary: We propose a method to reconstruct activations of classification and semantic segmentation networks by stitching them with a GAN generator utilizing a 1x1 convolution. We test our approach on images of animals from the AFHQ wild dataset, ImageNet1K, and real-world digital pathology scans of stained tissue samples.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work, we propose a fast and accurate method to reconstruct activations of classification and semantic segmentation networks by stitching them with a GAN generator utilizing a 1x1 convolution. We test our approach on images of animals from the AFHQ wild dataset, ImageNet1K, and real-world digital pathology scans of stained tissue samples. Our results show comparable performance to established gradient descent methods but with a processing time that is two orders of magnitude faster, making this approach promising for practical applications.

Related papers

• Time Step Generating: A Universal Synthesized Deepfake Image Detector [0.4488895231267077]
  We propose a universal synthetic image detector Time Step Generating (TSG) TSG does not rely on pre-trained models' reconstructing ability, specific datasets, or sampling algorithms. We test the proposed TSG on the large-scale GenImage benchmark and it achieves significant improvements in both accuracy and generalizability.
  arXiv  Detail & Related papers  (2024-11-17T09:39:50Z)
• Supervised Homography Learning with Realistic Dataset Generation [60.934401870005026]
  We propose an iterative framework, which consists of two phases: a generation phase and a training phase. In the generation phase, given an unlabeled image pair, we utilize the pre-estimated dominant plane masks and homography of the pair. In the training phase, the generated data is used to train the supervised homography network.
  arXiv  Detail & Related papers  (2023-07-28T07:03:18Z)
• Unsupervised Domain Transfer with Conditional Invertible Neural Networks [83.90291882730925]
  We propose a domain transfer approach based on conditional invertible neural networks (cINNs) Our method inherently guarantees cycle consistency through its invertible architecture, and network training can efficiently be conducted with maximum likelihood. Our method enables the generation of realistic spectral data and outperforms the state of the art on two downstream classification tasks.
  arXiv  Detail & Related papers  (2023-03-17T18:00:27Z)
• Ensembling with Deep Generative Views [72.70801582346344]
  generative models can synthesize "views" of artificial images that mimic real-world variations, such as changes in color or pose. Here, we investigate whether such views can be applied to real images to benefit downstream analysis tasks such as image classification. We use StyleGAN2 as the source of generative augmentations and investigate this setup on classification tasks involving facial attributes, cat faces, and cars.
  arXiv  Detail & Related papers  (2021-04-29T17:58:35Z)
• Randomized Histogram Matching: A Simple Augmentation for Unsupervised Domain Adaptation in Overhead Imagery [3.187381965457262]
  We propose a fast real-time unsupervised training augmentation technique, termed randomized histogram matching (RHM) RHM consistently yields similar or superior performance compared to state-of-the-art unsupervised domain adaptation approaches. RHM also offers substantially better performance than other comparably simple approaches that are widely used for overhead imagery.
  arXiv  Detail & Related papers  (2021-04-28T21:59:54Z)
• Time Series Imaging for Link Layer Anomaly Classification in Wireless Networks [0.6015898117103068]
  In this paper, we perform a first time analysis of image-based representation techniques for wireless anomaly detection. We propose a new deep learning architecture enabling accurate anomaly detection. Our results demonstrate the potential of transformation of time series signals to images to improve classification performance.
  arXiv  Detail & Related papers  (2021-04-02T10:23:06Z)
• Guiding GANs: How to control non-conditional pre-trained GANs for conditional image generation [69.10717733870575]
  We present a novel method for guiding generic non-conditional GANs to behave as conditional GANs. Our approach adds into the mix an encoder network to generate the high-dimensional random input that are fed to the generator network of a non-conditional GAN.
  arXiv  Detail & Related papers  (2021-01-04T14:03:32Z)
• Encoding large scale cosmological structure with Generative Adversarial Networks [0.0]
  We use a trained GAN to construct a simple autoencoder (AE) as a first step towards building a predictive model. Both the GAN and AE are trained on images issued from two types of N-body simulations, namely 2D and 3D simulations. We find that the GAN successfully generates new images that are statistically consistent with the images it was trained on.
  arXiv  Detail & Related papers  (2020-11-10T17:11:14Z)
• Temporal Attention-Augmented Graph Convolutional Network for Efficient Skeleton-Based Human Action Recognition [97.14064057840089]
  Graphal networks (GCNs) have been very successful in modeling non-Euclidean data structures. Most GCN-based action recognition methods use deep feed-forward networks with high computational complexity to process all skeletons in an action. We propose a temporal attention module (TAM) for increasing the efficiency in skeleton-based action recognition.
  arXiv  Detail & Related papers  (2020-10-23T08:01:55Z)
• Large-scale spatiotemporal photonic reservoir computer for image classification [0.8701566919381222]
  We propose a scalable photonic architecture for implementation of feedforward and recurrent neural networks to perform the classification of handwritten digits. Our experiment exploits off-the-shelf optical and electronic components to currently achieve a network size of 16,384 nodes.
  arXiv  Detail & Related papers  (2020-04-06T10:22:31Z)
• Unlimited Resolution Image Generation with R2D2-GANs [69.90258455164513]
  We present a novel simulation technique for generating high quality images of any predefined resolution. This method can be used to synthesize sonar scans of size equivalent to those collected during a full-length mission. The data produced is continuous, realistically-looking, and can also be generated at least two times faster than the real speed of acquisition.
  arXiv  Detail & Related papers  (2020-03-02T17:49:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.