Continual Learning for Visual Search with Backward Consistent Feature
Embedding
- URL: http://arxiv.org/abs/2205.13384v1
- Date: Thu, 26 May 2022 14:15:29 GMT
- Title: Continual Learning for Visual Search with Backward Consistent Feature
Embedding
- Authors: Timmy S. T. Wan, Jun-Cheng Chen, Tzer-Yi Wu, Chu-Song Chen
- Abstract summary: In visual search, the gallery set could be incrementally growing and added to the database in practice.
Existing methods rely on the model trained on the entire dataset, ignoring the continual updating of the model.
We introduce a continual learning (CL) approach that can handle the incrementally growing gallery set with backward embedding consistency.
- Score: 26.89922800367714
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In visual search, the gallery set could be incrementally growing and added to
the database in practice. However, existing methods rely on the model trained
on the entire dataset, ignoring the continual updating of the model. Besides,
as the model updates, the new model must re-extract features for the entire
gallery set to maintain compatible feature space, imposing a high computational
cost for a large gallery set. To address the issues of long-term visual search,
we introduce a continual learning (CL) approach that can handle the
incrementally growing gallery set with backward embedding consistency. We
enforce the losses of inter-session data coherence, neighbor-session model
coherence, and intra-session discrimination to conduct a continual learner. In
addition to the disjoint setup, our CL solution also tackles the situation of
increasingly adding new classes for the blurry boundary without assuming all
categories known in the beginning and during model update. To our knowledge,
this is the first CL method both tackling the issue of backward-consistent
feature embedding and allowing novel classes to occur in the new sessions.
Extensive experiments on various benchmarks show the efficacy of our approach
under a wide range of setups.
Related papers
- OrCo: Towards Better Generalization via Orthogonality and Contrast for Few-Shot Class-Incremental Learning [57.43911113915546]
Few-Shot Class-Incremental Learning (FSCIL) introduces a paradigm in which the problem space expands with limited data.
FSCIL methods inherently face the challenge of catastrophic forgetting as data arrives incrementally.
We propose the OrCo framework built on two core principles: features' orthogonality in the representation space, and contrastive learning.
arXiv Detail & Related papers (2024-03-27T13:30:48Z) - Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters [65.15700861265432]
We present a parameter-efficient continual learning framework to alleviate long-term forgetting in incremental learning with vision-language models.
Our approach involves the dynamic expansion of a pre-trained CLIP model, through the integration of Mixture-of-Experts (MoE) adapters.
To preserve the zero-shot recognition capability of vision-language models, we introduce a Distribution Discriminative Auto-Selector.
arXiv Detail & Related papers (2024-03-18T08:00:23Z) - Enhanced Few-Shot Class-Incremental Learning via Ensemble Models [34.84881941101568]
Few-shot class-incremental learning aims to continually fit new classes with limited training data.
The main challenges are overfitting the rare new training samples and forgetting old classes.
We propose a new ensemble model framework cooperating with data augmentation to boost generalization.
arXiv Detail & Related papers (2024-01-14T06:07:07Z) - Incremental Object Detection with CLIP [36.478530086163744]
We propose a visual-language model such as CLIP to generate text feature embeddings for different class sets.
We then employ super-classes to replace the unavailable novel classes in the early learning stage to simulate the incremental scenario.
We incorporate the finely recognized detection boxes as pseudo-annotations into the training process, thereby further improving the detection performance.
arXiv Detail & Related papers (2023-10-13T01:59:39Z) - Multi-View Class Incremental Learning [57.14644913531313]
Multi-view learning (MVL) has gained great success in integrating information from multiple perspectives of a dataset to improve downstream task performance.
This paper investigates a novel paradigm called multi-view class incremental learning (MVCIL), where a single model incrementally classifies new classes from a continual stream of views.
arXiv Detail & Related papers (2023-06-16T08:13:41Z) - Retrieval-Enhanced Contrastive Vision-Text Models [61.783728119255365]
We propose to equip vision-text models with the ability to refine their embedding with cross-modal retrieved information from a memory at inference time.
Remarkably, we show that this can be done with a light-weight, single-layer, fusion transformer on top of a frozen CLIP.
Our experiments validate that our retrieval-enhanced contrastive (RECO) training improves CLIP performance substantially on several challenging fine-grained tasks.
arXiv Detail & Related papers (2023-06-12T15:52:02Z) - Few-Shot Class-Incremental Learning by Sampling Multi-Phase Tasks [59.12108527904171]
A model should recognize new classes and maintain discriminability over old classes.
The task of recognizing few-shot new classes without forgetting old classes is called few-shot class-incremental learning (FSCIL)
We propose a new paradigm for FSCIL based on meta-learning by LearnIng Multi-phase Incremental Tasks (LIMIT)
arXiv Detail & Related papers (2022-03-31T13:46:41Z) - Towards Universal Backward-Compatible Representation Learning [29.77801805854168]
backward-compatible representation learning is introduced to support backfill-free model upgrades.
We first introduce a new problem of universal backward-compatible representation learning, covering all possible data split in model upgrades.
We propose a simple yet effective method, dubbed Universal Backward- Training (UniBCT) with a novel structural prototype refinement algorithm.
arXiv Detail & Related papers (2022-03-03T09:23:51Z) - vCLIMB: A Novel Video Class Incremental Learning Benchmark [53.90485760679411]
We introduce vCLIMB, a novel video continual learning benchmark.
vCLIMB is a standardized test-bed to analyze catastrophic forgetting of deep models in video continual learning.
We propose a temporal consistency regularization that can be applied on top of memory-based continual learning methods.
arXiv Detail & Related papers (2022-01-23T22:14:17Z) - CoReS: Compatible Representations via Stationarity [20.607894099896214]
In visual search systems, compatible features enable the direct comparison of old and new learned features allowing to use them interchangeably over time.
We propose CoReS, a new training procedure to learn representations that are textitcompatible with those previously learned.
We demonstrate that our training procedure largely outperforms the current state of the art and is particularly effective in the case of multiple upgrades of the training-set.
arXiv Detail & Related papers (2021-11-15T09:35:54Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.