Learning Test-time Augmentation for Content-based Image Retrieval
- URL: http://arxiv.org/abs/2002.01642v5
- Date: Tue, 5 Jul 2022 04:28:32 GMT
- Title: Learning Test-time Augmentation for Content-based Image Retrieval
- Authors: Osman Tursun, Simon Denman, Sridha Sridharan and Clinton Fookes
- Abstract summary: Off-the-shelf convolutional neural network features achieve outstanding results in many image retrieval tasks.
Existing image retrieval approaches require fine-tuning or modification of pre-trained networks to adapt to variations unique to the target data.
Our method enhances the invariance of off-the-shelf features by aggregating features extracted from images augmented at test-time, with augmentations guided by a policy learned through reinforcement learning.
- Score: 42.188013259368766
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Off-the-shelf convolutional neural network features achieve outstanding
results in many image retrieval tasks. However, their invariance to target data
is pre-defined by the network architecture and training data. Existing image
retrieval approaches require fine-tuning or modification of pre-trained
networks to adapt to variations unique to the target data. In contrast, our
method enhances the invariance of off-the-shelf features by aggregating
features extracted from images augmented at test-time, with augmentations
guided by a policy learned through reinforcement learning. The learned policy
assigns different magnitudes and weights to the selected transformations, which
are selected from a list of image transformations. Policies are evaluated using
a metric learning protocol to learn the optimal policy. The model converges
quickly and the cost of each policy iteration is minimal as we propose an
off-line caching technique to greatly reduce the computational cost of
extracting features from augmented images. Experimental results on large
trademark retrieval (METU trademark dataset) and landmark retrieval (ROxford5k
and RParis6k scene datasets) tasks show that the learned ensemble of
transformations is highly effective for improving performance, and is
practical, and transferable.
Related papers
- Synergy and Diversity in CLIP: Enhancing Performance Through Adaptive Backbone Ensembling [58.50618448027103]
Contrastive Language-Image Pretraining (CLIP) stands out as a prominent method for image representation learning.
This paper explores the differences across various CLIP-trained vision backbones.
Method achieves a remarkable increase in accuracy of up to 39.1% over the best single backbone.
arXiv Detail & Related papers (2024-05-27T12:59:35Z) - Image edge enhancement for effective image classification [7.470763273994321]
We propose an edge enhancement-based method to enhance both accuracy and training speed of neural networks.
Our approach involves extracting high frequency features, such as edges, from images within the available dataset and fusing them with the original images.
arXiv Detail & Related papers (2024-01-13T10:01:34Z) - Augmentation Learning for Semi-Supervised Classification [13.519613713213277]
We propose a Semi-Supervised Learning method that automatically selects the most effective data augmentation policy for a particular dataset.
We show how policy learning can be used to adapt augmentations to datasets beyond ImageNet.
arXiv Detail & Related papers (2022-08-03T10:06:51Z) - Combining Image Features and Patient Metadata to Enhance Transfer
Learning [0.0]
We compare the performance of six state-of-the-art deep neural networks in classification tasks when using only image features, to when these are combined with patient metadata.
Our results indicate that this performance enhancement may be a general property of deep networks and should be explored in other areas.
arXiv Detail & Related papers (2021-10-08T15:43:31Z) - AugNet: End-to-End Unsupervised Visual Representation Learning with
Image Augmentation [3.6790362352712873]
We propose AugNet, a new deep learning training paradigm to learn image features from a collection of unlabeled pictures.
Our experiments demonstrate that the method is able to represent the image in low dimensional space.
Unlike many deep-learning-based image retrieval algorithms, our approach does not require access to external annotated datasets.
arXiv Detail & Related papers (2021-06-11T09:02:30Z) - Contrastive Learning with Stronger Augmentations [63.42057690741711]
We propose a general framework called Contrastive Learning with Stronger Augmentations(A) to complement current contrastive learning approaches.
Here, the distribution divergence between the weakly and strongly augmented images over the representation bank is adopted to supervise the retrieval of strongly augmented queries.
Experiments showed the information from the strongly augmented images can significantly boost the performance.
arXiv Detail & Related papers (2021-04-15T18:40:04Z) - Learning Representational Invariances for Data-Efficient Action
Recognition [52.23716087656834]
We show that our data augmentation strategy leads to promising performance on the Kinetics-100, UCF-101, and HMDB-51 datasets.
We also validate our data augmentation strategy in the fully supervised setting and demonstrate improved performance.
arXiv Detail & Related papers (2021-03-30T17:59:49Z) - MetaAugment: Sample-Aware Data Augmentation Policy Learning [20.988767360529362]
We learn a sample-aware data augmentation policy efficiently by formulating it as a sample reweighting problem.
An augmentation policy network takes a transformation and the corresponding augmented image as inputs, and outputs a weight to adjust the augmented image loss computed by a task network.
At training stage, the task network minimizes the weighted losses of augmented training images, while the policy network minimizes the loss of the task network on a validation set via meta-learning.
arXiv Detail & Related papers (2020-12-22T15:19:27Z) - Data Augmentation for Meta-Learning [58.47185740820304]
meta-learning algorithms sample data, query data, and tasks on each training step.
Data augmentation can be used not only to expand the number of images available per class, but also to generate entirely new classes/tasks.
Our proposed meta-specific data augmentation significantly improves the performance of meta-learners on few-shot classification benchmarks.
arXiv Detail & Related papers (2020-10-14T13:48:22Z) - FeatMatch: Feature-Based Augmentation for Semi-Supervised Learning [64.32306537419498]
We propose a novel learned feature-based refinement and augmentation method that produces a varied set of complex transformations.
These transformations also use information from both within-class and across-class representations that we extract through clustering.
We demonstrate that our method is comparable to current state of art for smaller datasets while being able to scale up to larger datasets.
arXiv Detail & Related papers (2020-07-16T17:55:31Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.