Online Backfilling with No Regret for Large-Scale Image Retrieval
- URL: http://arxiv.org/abs/2301.03767v1
- Date: Tue, 10 Jan 2023 03:10:32 GMT
- Title: Online Backfilling with No Regret for Large-Scale Image Retrieval
- Authors: Seonguk Seo, Mustafa Gokhan Uzunbas, Bohyung Han, Sara Cao, Joena
Zhang, Taipeng Tian, Ser-Nam Lim
- Abstract summary: Backfilling is the process of re-extracting all gallery embeddings from upgraded models in image retrieval systems.
We propose an online backfilling algorithm, which enables us to achieve a progressive performance improvement during the backfilling process.
We incorporate a reverse transformation module for more effective and efficient merging, which is further enhanced by adopting a metric-compatible contrastive learning approach.
- Score: 50.162438586686356
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Backfilling is the process of re-extracting all gallery embeddings from
upgraded models in image retrieval systems. It inevitably requires a
prohibitively large amount of computational cost and even entails the downtime
of the service. Although backward-compatible learning sidesteps this challenge
by tackling query-side representations, this leads to suboptimal solutions in
principle because gallery embeddings cannot benefit from model upgrades. We
address this dilemma by introducing an online backfilling algorithm, which
enables us to achieve a progressive performance improvement during the
backfilling process while not sacrificing the final performance of new model
after the completion of backfilling. To this end, we first propose a simple
distance rank merge technique for online backfilling. Then, we incorporate a
reverse transformation module for more effective and efficient merging, which
is further enhanced by adopting a metric-compatible contrastive learning
approach. These two components help to make the distances of old and new models
compatible, resulting in desirable merge results during backfilling with no
extra computational overhead. Extensive experiments show the effectiveness of
our framework on four standard benchmarks in various settings.
Related papers
- Any Image Restoration with Efficient Automatic Degradation Adaptation [132.81912195537433]
We propose a unified manner to achieve joint embedding by leveraging the inherent similarities across various degradations for efficient and comprehensive restoration.
Our network sets new SOTA records while reducing model complexity by approximately -82% in trainable parameters and -85% in FLOPs.
arXiv Detail & Related papers (2024-07-18T10:26:53Z) - Unified-Width Adaptive Dynamic Network for All-In-One Image Restoration [50.81374327480445]
We introduce a novel concept positing that intricate image degradation can be represented in terms of elementary degradation.
We propose the Unified-Width Adaptive Dynamic Network (U-WADN), consisting of two pivotal components: a Width Adaptive Backbone (WAB) and a Width Selector (WS)
The proposed U-WADN achieves better performance while simultaneously reducing up to 32.3% of FLOPs and providing approximately 15.7% real-time acceleration.
arXiv Detail & Related papers (2024-01-24T04:25:12Z) - AdaptIR: Parameter Efficient Multi-task Adaptation for Pre-trained Image
Restoration Models [58.10797482129863]
We propose AdaptIR, a novel parameter efficient transfer learning method for adapting pre-trained restoration models.
Experiments demonstrate that the proposed method can achieve comparable or even better performance than full fine-tuning, while only using 0.6%.
arXiv Detail & Related papers (2023-12-12T14:27:59Z) - Rethinking PRL: A Multiscale Progressively Residual Learning Network for
Inverse Halftoning [3.632876183725243]
inverse halftoning is a classic image restoration task, aiming to recover continuous-tone images from halftone images with only bilevel pixels.
We propose an end-to-end multiscale progressively residual learning network (MSPRL), which has a UNet architecture and takes multiscale input images.
arXiv Detail & Related papers (2023-05-27T03:37:33Z) - FastFill: Efficient Compatible Model Update [40.27741553705222]
FastFill is a compatible model update process using feature alignment and policy based partial backfilling.
We show that previous backfilling strategies suffer from decreased performance and demonstrate the importance of both the training objective and the ordering in online partial backfilling.
arXiv Detail & Related papers (2023-03-08T18:03:51Z) - Towards Universal Backward-Compatible Representation Learning [29.77801805854168]
backward-compatible representation learning is introduced to support backfill-free model upgrades.
We first introduce a new problem of universal backward-compatible representation learning, covering all possible data split in model upgrades.
We propose a simple yet effective method, dubbed Universal Backward- Training (UniBCT) with a novel structural prototype refinement algorithm.
arXiv Detail & Related papers (2022-03-03T09:23:51Z) - Hot-Refresh Model Upgrades with Regression-Alleviating Compatible
Training in Image Retrieval [34.84329831602699]
cold-refresh model upgrades can only deploy new models after the gallery is overall backfilled, taking weeks or even months for massive data.
In contrast, hot-refresh model upgrades deploy the new model immediately and then gradually improve the retrieval accuracy by backfilling the gallery on-the-fly.
arXiv Detail & Related papers (2022-01-24T14:59:12Z) - Plug-and-Play Image Restoration with Deep Denoiser Prior [186.84724418955054]
We show that a denoiser can implicitly serve as the image prior for model-based methods to solve many inverse problems.
We set up a benchmark deep denoiser prior by training a highly flexible and effective CNN denoiser.
We then plug the deep denoiser prior as a modular part into a half quadratic splitting based iterative algorithm to solve various image restoration problems.
arXiv Detail & Related papers (2020-08-31T17:18:58Z) - The Power of Triply Complementary Priors for Image Compressive Sensing [89.14144796591685]
We propose a joint low-rank deep (LRD) image model, which contains a pair of complementaryly trip priors.
We then propose a novel hybrid plug-and-play framework based on the LRD model for image CS.
To make the optimization tractable, a simple yet effective algorithm is proposed to solve the proposed H-based image CS problem.
arXiv Detail & Related papers (2020-05-16T08:17:44Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.