Towards Composable Distributions of Latent Space Augmentations

• URL: http://arxiv.org/abs/2303.03462v1
• Date: Mon, 6 Mar 2023 19:37:01 GMT
• Title: Towards Composable Distributions of Latent Space Augmentations
• Authors: Omead Pooladzandi, Jeffrey Jiang, Sunay Bhat, Gregory Pottie
• Abstract summary: We propose a composable framework for latent space image augmentation that allows for easy combination of multiple augmentations. Our framework is based on the Variational Autoencoder architecture and uses a novel approach for augmentation via linear transformation within the latent space itself. We show these properties are better performing with certain pairs of augmentations, but we can transfer the latent space to other sets of augmentations to modify performance.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a composable framework for latent space image augmentation that allows for easy combination of multiple augmentations. Image augmentation has been shown to be an effective technique for improving the performance of a wide variety of image classification and generation tasks. Our framework is based on the Variational Autoencoder architecture and uses a novel approach for augmentation via linear transformation within the latent space itself. We explore losses and augmentation latent geometry to enforce the transformations to be composable and involuntary, thus allowing the transformations to be readily combined or inverted. Finally, we show these properties are better performing with certain pairs of augmentations, but we can transfer the latent space to other sets of augmentations to modify performance, effectively constraining the VAE's bottleneck to preserve the variance of specific augmentations and features of the image which we care about. We demonstrate the effectiveness of our approach with initial results on the MNIST dataset against both a standard VAE and a Conditional VAE. This latent augmentation method allows for much greater control and geometric interpretability of the latent space, making it a valuable tool for researchers and practitioners in the field.

Related papers

• Semantic-Aligned Adversarial Evolution Triangle for High-Transferability Vision-Language Attack [51.16384207202798]
  Vision-language pre-training models are vulnerable to multimodal adversarial examples (AEs) Previous approaches augment image-text pairs to enhance diversity within the adversarial example generation process. We propose sampling from adversarial evolution triangles composed of clean, historical, and current adversarial examples to enhance adversarial diversity.
  arXiv  Detail & Related papers  (2024-11-04T23:07:51Z)
• HAT: Hybrid Attention Transformer for Image Restoration [61.74223315807691]
  Transformer-based methods have shown impressive performance in image restoration tasks, such as image super-resolution and denoising. We propose a new Hybrid Attention Transformer (HAT) to activate more input pixels for better restoration. Our HAT achieves state-of-the-art performance both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2023-09-11T05:17:55Z)
• LLIC: Large Receptive Field Transform Coding with Adaptive Weights for Learned Image Compression [27.02281402358164]
  We propose Large Receptive Field Transform Coding with Adaptive Weights for Learned Image Compression. We introduce a few large kernelbased depth-wise convolutions to reduce more redundancy while maintaining modest complexity. Our LLIC models achieve state-of-the-art performances and better trade-offs between performance and complexity.
  arXiv  Detail & Related papers  (2023-04-19T11:19:10Z)
• Local Magnification for Data and Feature Augmentation [53.04028225837681]
  We propose an easy-to-implement and model-free data augmentation method called Local Magnification (LOMA) LOMA generates additional training data by randomly magnifying a local area of the image. Experiments show that our proposed LOMA, though straightforward, can be combined with standard data augmentation to significantly improve the performance on image classification and object detection.
  arXiv  Detail & Related papers  (2022-11-15T02:51:59Z)
• Activating More Pixels in Image Super-Resolution Transformer [53.87533738125943]
  Transformer-based methods have shown impressive performance in low-level vision tasks, such as image super-resolution. We propose a novel Hybrid Attention Transformer (HAT) to activate more input pixels for better reconstruction. Our overall method significantly outperforms the state-of-the-art methods by more than 1dB.
  arXiv  Detail & Related papers  (2022-05-09T17:36:58Z)
• Plug-In Inversion: Model-Agnostic Inversion for Vision with Data Augmentations [61.95114821573875]
  We introduce Plug-In Inversion, which relies on a simple set of augmentations and does not require excessive hyper- parameter tuning. We illustrate the practicality of our approach by inverting Vision Transformers (ViTs) and Multi-Layer Perceptrons (MLPs) trained on the ImageNet dataset.
  arXiv  Detail & Related papers  (2022-01-31T02:12:45Z)
• Analogous to Evolutionary Algorithm: Designing a Unified Sequence Model [58.17021225930069]
  We explain the rationality of Vision Transformer by analogy with the proven practical Evolutionary Algorithm (EA) We propose a more efficient EAT model, and design task-related heads to deal with different tasks more flexibly. Our approach achieves state-of-the-art results on the ImageNet classification task compared with recent vision transformer works.
  arXiv  Detail & Related papers  (2021-05-31T16:20:03Z)
• Group Equivariant Generative Adversarial Networks [7.734726150561089]
  In this work, we explicitly incorporate inductive symmetry priors into the network architectures via group-equivariant convolutional networks. Group-convariants have higher expressive power with fewer samples and lead to better gradient feedback between generator and discriminator.
  arXiv  Detail & Related papers  (2020-05-04T17:38:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.