AlignMix: Improving representation by interpolating aligned features

• URL: http://arxiv.org/abs/2103.15375v1
• Date: Mon, 29 Mar 2021 07:03:18 GMT
• Title: AlignMix: Improving representation by interpolating aligned features
• Authors: Shashanka Venkataramanan, Yannis Avrithis, Ewa Kijak, Laurent Amsaleg
• Abstract summary: We introduce AlignMix, where we geometrically align two images in the feature space. We show that an autoencoder can still improve representation learning under mixup, without the ever seeing decoded images.
• Score: 19.465347590276114
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Mixup is a powerful data augmentation method that interpolates between two or more examples in the input or feature space and between the corresponding target labels. Many recent mixup methods focus on cutting and pasting two or more objects into one image, which is more about efficient processing than interpolation. However, how to best interpolate images is not well defined. In this sense, mixup has been connected to autoencoders, because often autoencoders "interpolate well", for instance generating an image that continuously deforms into another. In this work, we revisit mixup from the interpolation perspective and introduce AlignMix, where we geometrically align two images in the feature space. The correspondences allow us to interpolate between two sets of features, while keeping the locations of one set. Interestingly, this gives rise to a situation where mixup retains mostly the geometry or pose of one image and the texture of the other, connecting it to style transfer. More than that, we show that an autoencoder can still improve representation learning under mixup, without the classifier ever seeing decoded images. AlignMix outperforms state-of-the-art mixup methods on five different benchmarks.

Related papers

• AID: Attention Interpolation of Text-to-Image Diffusion [64.87754163416241]
  We introduce a training-free technique named Attention Interpolation via Diffusion (AID) AID fuses the interpolated attention with self-attention to boost fidelity. We also present a variant, Conditional-guided Attention Interpolation via Diffusion (AID), that considers as a condition-dependent generative process.
  arXiv  Detail & Related papers  (2024-03-26T17:57:05Z)
• Adversarial AutoMixup [50.1874436169571]
  We propose AdAutomixup, an adversarial automatic mixup augmentation approach. It generates challenging samples to train a robust classifier for image classification. Our approach outperforms the state of the art in various classification scenarios.
  arXiv  Detail & Related papers  (2023-12-19T08:55:00Z)
• SpliceMix: A Cross-scale and Semantic Blending Augmentation Strategy for Multi-label Image Classification [46.8141860303439]
  We introduce a simple but effective augmentation strategy for multi-label image classification, namely SpliceMix. The "splice" in our method is two-fold: 1) Each mixed image is a splice of several downsampled images in the form of a grid, where the semantics of images attending to mixing are blended without object deficiencies for alleviating co-occurred bias; 2) We splice mixed images and the original mini-batch to form a new SpliceMixed mini-batch, which allows an image with different scales to contribute to training together.
  arXiv  Detail & Related papers  (2023-11-26T05:45:27Z)
• Multi-interactive Feature Learning and a Full-time Multi-modality Benchmark for Image Fusion and Segmentation [66.15246197473897]
  Multi-modality image fusion and segmentation play a vital role in autonomous driving and robotic operation. We propose a textbfMulti-textbfinteractive textbfFeature learning architecture for image fusion and textbfSegmentation.
  arXiv  Detail & Related papers  (2023-08-04T01:03:58Z)
• GuidedMixup: An Efficient Mixup Strategy Guided by Saliency Maps [6.396288020763144]
  We propose GuidedMixup, which aims to retain the salient regions in mixup images with low computational overhead. We develop an efficient pairing algorithm that pursues to minimize the conflict of salient regions of paired images. Experiments on several datasets demonstrate that GuidedMixup provides a good trade-off between augmentation overhead and generalization performance.
  arXiv  Detail & Related papers  (2023-06-29T00:55:51Z)
• Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondence [88.00004819064672]
  Diffusion Hyperfeatures is a framework for consolidating multi-scale and multi-timestep feature maps into per-pixel feature descriptors. Our method achieves superior performance on the SPair-71k real image benchmark.
  arXiv  Detail & Related papers  (2023-05-23T17:58:05Z)
• Semantic Image Fusion [2.4366811507669124]
  This paper proposes a novel system for the semantic combination of visual content using pre-trained CNN network architectures. Simple "choose maximum" and "local majority" filter based fusion rules are utilised for feature map fusion. The developed methods are able to give equivalent low-level fusion performance to state of the art methods.
  arXiv  Detail & Related papers  (2021-10-13T13:15:16Z)
• Bridging the Visual Gap: Wide-Range Image Blending [16.464837892640812]
  We introduce an effective deep-learning model to realize wide-range image blending. We experimentally demonstrate that our proposed method is able to produce visually appealing results.
  arXiv  Detail & Related papers  (2021-03-28T15:07:45Z)
• SnapMix: Semantically Proportional Mixing for Augmenting Fine-grained Data [124.95585891086894]
  Proposal is called Semantically Proportional Mixing (SnapMix) It exploits class activation map (CAM) to lessen the label noise in augmenting fine-grained data. Our method consistently outperforms existing mixed-based approaches.
  arXiv  Detail & Related papers  (2020-12-09T03:37:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.