DeDrift: Robust Similarity Search under Content Drift

• URL: http://arxiv.org/abs/2308.02752v1
• Date: Sat, 5 Aug 2023 00:12:39 GMT
• Title: DeDrift: Robust Similarity Search under Content Drift
• Authors: Dmitry Baranchuk, Matthijs Douze, Yash Upadhyay, I. Zeki Yalniz
• Abstract summary: We introduce and analyze real-world image and video datasets for which temporal information is available over a long time period. Based on the learnings, we devise DeDrift, a method that updates embedding quantizers to continuously adapt large-scale indexing structures on-the-fly. DeDrift almost eliminates the accuracy degradation due to the query and database content drift while being up to 100x faster than a full index reconstruction.
• Score: 13.127769273549042
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The statistical distribution of content uploaded and searched on media sharing sites changes over time due to seasonal, sociological and technical factors. We investigate the impact of this "content drift" for large-scale similarity search tools, based on nearest neighbor search in embedding space. Unless a costly index reconstruction is performed frequently, content drift degrades the search accuracy and efficiency. The degradation is especially severe since, in general, both the query and database distributions change. We introduce and analyze real-world image and video datasets for which temporal information is available over a long time period. Based on the learnings, we devise DeDrift, a method that updates embedding quantizers to continuously adapt large-scale indexing structures on-the-fly. DeDrift almost eliminates the accuracy degradation due to the query and database content drift while being up to 100x faster than a full index reconstruction.

Related papers

• Forward Index Compression for Learned Sparse Retrieval [15.629655228398567]
  We focus on the size of a data structure that is common to all algorithmic flavors and that constitutes a substantial fraction of the overall index size: the forward index.<n>In particular, we seek compression techniques to reduce the storage footprint of the forward index without compromising search quality or inner product computation latency.
  arXiv  Detail & Related papers  (2026-02-05T08:35:17Z)
• What You Have is What You Track: Adaptive and Robust Multimodal Tracking [72.92244578461869]
  We present the first comprehensive study on tracker performance with temporally incomplete multimodal data.<n>Our model achieves SOTA performance across 9 benchmarks, excelling in both conventional complete and missing modality settings.
  arXiv  Detail & Related papers  (2025-07-08T11:40:21Z)
• Query Drift Compensation: Enabling Compatibility in Continual Learning of Retrieval Embedding Models [12.586519025284328]
  We study how the already indexed corpus can still be effectively used without the need of re-indexing.<n>We employ embedding distillation on both query and document embeddings to maintain stability.<n>We propose a novel query drift compensation method during retrieval to project new model query embeddings to the old embedding space.
  arXiv  Detail & Related papers  (2025-05-27T14:52:52Z)
• A Lightweight Moment Retrieval System with Global Re-Ranking and Robust Adaptive Bidirectional Temporal Search [3.4271696759611068]
  The exponential growth of digital video content has posed critical challenges in moment-level video retrieval. Current retrieval systems are constrained by computational inefficiencies, temporal context limitations, and the intrinsic complexity of navigating video content.
  arXiv  Detail & Related papers  (2025-04-12T17:49:46Z)
• VoD: Learning Volume of Differences for Video-Based Deepfake Detection [9.407035514709293]
  This paper introduces a novel Deepfake detention framework, Volume of Differences (VoD) VoD is designed to enhance detection accuracy by exploiting temporal and spatial inconsistencies between consecutive video frames. We evaluate our approach with intra-dataset and cross-dataset testing scenarios on various well-known Deepfake datasets.
  arXiv  Detail & Related papers  (2025-03-10T17:59:38Z)
• RCRank: Multimodal Ranking of Root Causes of Slow Queries in Cloud Database Systems [38.72325043937881]
  Root-cause diagnosis plays an indispensable role in facilitating slow-query detection and revision. This paper proposes a method capable of both identifying possible root cause types for slow queries and ranking these according to their potential for accelerating slow queries. To enable more accurate and detailed diagnoses, we propose the multimodal Ranking for the Root Causes of slow queries (RCRank) framework.
  arXiv  Detail & Related papers  (2025-03-06T09:35:20Z)
• Investigating the Scalability of Approximate Sparse Retrieval Algorithms to Massive Datasets [8.1990111961557]
  We investigate the behavior of state-of-the-art retrieval algorithms on massive datasets. We compare and contrast the recently-proposed Seismic and graph-based solutions adapted from dense retrieval. We extensively evaluate Splade embeddings of 138M passages from MsMarco-v2 and report indexing time and other efficiency and effectiveness metrics.
  arXiv  Detail & Related papers  (2025-01-20T17:59:21Z)
• On the Consistency of Video Large Language Models in Temporal Comprehension [57.985769348320616]
  Video large language models (Video-LLMs) can temporally ground language queries and retrieve video moments. We conduct a study on prediction consistency -- a key indicator for robustness and trustworthiness of temporal grounding.
  arXiv  Detail & Related papers  (2024-11-20T00:47:17Z)
• WindTunnel -- A Framework for Community Aware Sampling of Large Corpora [0.0]
  WindTunnel is a framework developed at Yext to generate representative samples of large corpora. WindTunnel overcomes limitations in current sampling methods, providing more accurate evaluations.
  arXiv  Detail & Related papers  (2024-10-27T00:49:52Z)
• Semi-Parametric Retrieval via Binary Token Index [71.78109794895065]
  Semi-parametric Vocabulary Disentangled Retrieval (SVDR) is a novel semi-parametric retrieval framework. It supports two types of indexes: an embedding-based index for high effectiveness, akin to existing neural retrieval methods; and a binary token index that allows for quick and cost-effective setup, resembling traditional term-based retrieval. It achieves a 3% higher top-1 retrieval accuracy compared to the dense retriever DPR when using an embedding-based index and a 9% higher top-1 accuracy compared to BM25 when using a binary token index.
  arXiv  Detail & Related papers  (2024-05-03T08:34:13Z)
• Methods for Generating Drift in Text Streams [49.3179290313959]
  Concept drift is a frequent phenomenon in real-world datasets and corresponds to changes in data distribution over time. This paper provides four textual drift generation methods to ease the production of datasets with labeled drifts. Results show that all methods have their performance degraded right after the drifts, and the incremental SVM is the fastest to run and recover the previous performance levels.
  arXiv  Detail & Related papers  (2024-03-18T23:48:33Z)
• Spatial Decomposition and Temporal Fusion based Inter Prediction for Learned Video Compression [59.632286735304156]
  We propose a spatial decomposition and temporal fusion based inter prediction for learned video compression. With the SDD-based motion model and long short-term temporal fusion, our proposed learned video can obtain more accurate inter prediction contexts.
  arXiv  Detail & Related papers  (2024-01-29T03:30:21Z)
• Toward Accurate and Temporally Consistent Video Restoration from Raw Data [20.430231283171327]
  We present a new VJDD framework by consistent and accurate latent space propagation. The proposed losses can circumvent the error accumulation problem caused by inaccurate flow estimation. Experiments demonstrate the leading VJDD performance in term of restoration accuracy, perceptual quality and temporal consistency.
  arXiv  Detail & Related papers  (2023-12-25T12:38:03Z)
• Temporally Consistent Transformers for Video Generation [80.45230642225913]
  To generate accurate videos, algorithms have to understand the spatial and temporal dependencies in the world. No established benchmarks on complex data exist for rigorously evaluating video generation with long temporal dependencies. We introduce the Temporally Consistent Transformer (TECO), a generative model that substantially improves long-term consistency while also reducing sampling time.
  arXiv  Detail & Related papers  (2022-10-05T17:15:10Z)
• Mathematical Models for Local Sensing Hashes [7.400475825464313]
  We show that approximated index structures offer a good opportunity to accelerate the neighbor search for clustering and outlier detection. We indicate directions to mathematically model the properties of local sensing hashes.
  arXiv  Detail & Related papers  (2021-11-16T10:40:55Z)
• Deconfounded Video Moment Retrieval with Causal Intervention [80.90604360072831]
  We tackle the task of video moment retrieval (VMR), which aims to localize a specific moment in a video according to a textual query. Existing methods primarily model the matching relationship between query and moment by complex cross-modal interactions. We propose a causality-inspired VMR framework that builds structural causal model to capture the true effect of query and video content on the prediction.
  arXiv  Detail & Related papers  (2021-06-03T01:33:26Z)
• Post-Estimation Smoothing: A Simple Baseline for Learning with Side Information [102.18616819054368]
  We propose a post-estimation smoothing operator as a fast and effective method for incorporating structural index data into prediction. Because the smoothing step is separate from the original predictor, it applies to a broad class of machine learning tasks. Our experiments on large scale spatial and temporal datasets highlight the speed and accuracy of post-estimation smoothing in practice.
  arXiv  Detail & Related papers  (2020-03-12T18:04:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.