Learning Filter-Aware Distance Metrics for Nearest Neighbor Search with Multiple Filters

• URL: http://arxiv.org/abs/2511.04073v1
• Date: Thu, 06 Nov 2025 05:24:41 GMT
• Title: Learning Filter-Aware Distance Metrics for Nearest Neighbor Search with Multiple Filters
• Authors: Ananya Sutradhar, Suryansh Gupta, Ravishankar Krishnaswamy, Haiyang Xu, Aseem Rastogi, Gopal Srinivasa,
• Abstract summary: Filtered Approximate Nearest Neighbor (ANN) search retrieves the closest vectors for a query vector from a dataset.<n>Existing graph-based methods typically incorporate filter awareness by assigning fixed penalties.<n>We propose a principled alternative that learns the optimal trade-off between vector distance and filter match directly from the data.
• Score: 7.493563348124023
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Filtered Approximate Nearest Neighbor (ANN) search retrieves the closest vectors for a query vector from a dataset. It enforces that a specified set of discrete labels $S$ for the query must be included in the labels of each retrieved vector. Existing graph-based methods typically incorporate filter awareness by assigning fixed penalties or prioritizing nodes based on filter satisfaction. However, since these methods use fixed, data in- dependent penalties, they often fail to generalize across datasets with diverse label and vector distributions. In this work, we propose a principled alternative that learns the optimal trade-off between vector distance and filter match directly from the data, rather than relying on fixed penalties. We formulate this as a constrained linear optimization problem, deriving weights that better reflect the underlying filter distribution and more effectively address the filtered ANN search problem. These learned weights guide both the search process and index construction, leading to graph structures that more effectively capture the underlying filter distribution and filter semantics. Our experiments demonstrate that adapting the distance function to the data significantly im- proves accuracy by 5-10% over fixed-penalty methods, providing a more flexible and generalizable framework for the filtered ANN search problem.

Related papers

• Efficient Filtered-ANN via Learning-based Query Planning [5.0924801385693454]
  We introduce a learning-based query planning framework that dynamically selects the most effective execution plan for each query.<n>Experiments show that our method achieves up to 4x acceleration with >= 90% recall comparing to the strong baselines.
  arXiv  Detail & Related papers  (2026-02-20T00:22:43Z)
• JAG: Joint Attribute Graphs for Filtered Nearest Neighbor Search [8.398505388767248]
  JAG (Joint Attribute Graphs) is a graph-based algorithm designed to deliver robust performance across the entire selectivity spectrum.<n>Our experimental results demonstrate that JAG significantly outperforms existing state-of-the-art baselines in both throughput and recall robustness.
  arXiv  Detail & Related papers  (2026-02-10T20:00:51Z)
• Attribute Filtering in Approximate Nearest Neighbor Search: An In-depth Experimental Study [18.5007917065799]
  We present a unified Filtering ANN search interface that encompasses the latest algorithms and evaluate them extensively from multiple perspectives.<n>First, we propose a comprehensive taxonomy of existing Filtering ANN algorithms based on attribute types and filtering strategies.<n>We then conduct a broad experimental evaluation on 10 algorithms and 12 methods across 4 datasets.
  arXiv  Detail & Related papers  (2025-08-22T09:54:57Z)
• Data Selection for Language Models via Importance Resampling [90.9263039747723]
  We formalize the problem of selecting a subset of a large raw unlabeled dataset to match a desired target distribution. We extend the classic importance resampling approach used in low-dimensions for LM data selection. We instantiate the DSIR framework with hashed n-gram features for efficiency, enabling the selection of 100M documents in 4.5 hours.
  arXiv  Detail & Related papers  (2023-02-06T23:57:56Z)
• Sparse Regularized Correlation Filter for UAV Object Tracking with adaptive Contextual Learning and Keyfilter Selection [20.786475337107472]
  correlation filter has been widely applied in unmanned aerial vehicle (UAV) tracking. It is fragile because of two inherent defects, i.e. boundary effect and filter corruption. We propose a novel $ell_1$ regularization correlation filter with adaptive contextual learning and keyfilter selection.
  arXiv  Detail & Related papers  (2022-05-07T10:25:56Z)
• Learning Versatile Convolution Filters for Efficient Visual Recognition [125.34595948003745]
  This paper introduces versatile filters to construct efficient convolutional neural networks. We conduct theoretical analysis on network complexity and an efficient convolution scheme is introduced. Experimental results on benchmark datasets and neural networks demonstrate that our versatile filters are able to achieve comparable accuracy as that of original filters.
  arXiv  Detail & Related papers  (2021-09-20T06:07:14Z)
• Message Passing in Graph Convolution Networks via Adaptive Filter Banks [81.12823274576274]
  We present a novel graph convolution operator, termed BankGCN. It decomposes multi-channel signals on graphs into subspaces and handles particular information in each subspace with an adapted filter. It achieves excellent performance in graph classification on a collection of benchmark graph datasets.
  arXiv  Detail & Related papers  (2021-06-18T04:23:34Z)
• Unsharp Mask Guided Filtering [53.14430987860308]
  The goal of this paper is guided image filtering, which emphasizes the importance of structure transfer during filtering. We propose a new and simplified formulation of the guided filter inspired by unsharp masking. Our formulation enjoys a filtering prior to a low-pass filter and enables explicit structure transfer by estimating a single coefficient.
  arXiv  Detail & Related papers  (2021-06-02T19:15:34Z)
• Equivalence of Correlation Filter and Convolution Filter in Visual Tracking [10.820122999766713]
  Correlation filter has been successfully applied to visual tracking. convolution filter is usually used for blurring, sharpening, embossing, edge detection, etc in image processing.
  arXiv  Detail & Related papers  (2021-05-01T04:05:37Z)
• Data Agnostic Filter Gating for Efficient Deep Networks [72.4615632234314]
  Current filter pruning methods mainly leverage feature maps to generate important scores for filters and prune those with smaller scores. In this paper, we propose a data filter pruning method that uses an auxiliary network named Dagger module to induce pruning. In addition, to help prune filters with certain FLOPs constraints, we leverage an explicit FLOPs-aware regularization to directly promote pruning filters toward target FLOPs.
  arXiv  Detail & Related papers  (2020-10-28T15:26:40Z)
• Dependency Aware Filter Pruning [74.69495455411987]
  Pruning a proportion of unimportant filters is an efficient way to mitigate the inference cost. Previous work prunes filters according to their weight norms or the corresponding batch-norm scaling factors. We propose a novel mechanism to dynamically control the sparsity-inducing regularization so as to achieve the desired sparsity.
  arXiv  Detail & Related papers  (2020-05-06T07:41:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.