Efficient Filtered-ANN via Learning-based Query Planning

• URL: http://arxiv.org/abs/2602.17914v1
• Date: Fri, 20 Feb 2026 00:22:43 GMT
• Title: Efficient Filtered-ANN via Learning-based Query Planning
• Authors: Zhuocheng Gan, Yifan Wang,
• Abstract summary: We introduce a learning-based query planning framework that dynamically selects the most effective execution plan for each query.<n>Experiments show that our method achieves up to 4x acceleration with >= 90% recall comparing to the strong baselines.
• Score: 5.0924801385693454
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Filtered ANN search is an increasingly important problem in vector retrieval, yet systems face a difficult trade-off due to the execution order: Pre-filtering (filtering first, then ANN over the passing subset) requires expensive per-predicate index construction, while post-filtering (ANN first, then filtering candidates) may waste computation and lose recall under low selectivity due to insufficient candidates after filtering. We introduce a learning-based query planning framework that dynamically selects the most effective execution plan for each query, using lightweight predictions derived from dataset and query statistics (e.g., dimensionality, corpus size, distribution features, and predicate statistics). The framework supports diverse filter types, including categorical/keyword and range predicates, and is generic to use any backend ANN index. Experiments show that our method achieves up to 4x acceleration with >= 90% recall comparing to the strong baselines.

Related papers

• Filtered Approximate Nearest Neighbor Search in Vector Databases: System Design and Performance Analysis [0.5249805590164902]
  Filtered Approximate Nearest Neighbor Search (FANNS) is used to combine semantic retrieval with metadata constraints.<n>In this work, we systematize the taxonomy of filtering strategies and evaluate their integration into vectors.<n>Our experiments reveal that algorithmic adaptations within the engine often override raw index performance.
  arXiv  Detail & Related papers  (2026-02-11T23:40:26Z)
• JAG: Joint Attribute Graphs for Filtered Nearest Neighbor Search [8.398505388767248]
  JAG (Joint Attribute Graphs) is a graph-based algorithm designed to deliver robust performance across the entire selectivity spectrum.<n>Our experimental results demonstrate that JAG significantly outperforms existing state-of-the-art baselines in both throughput and recall robustness.
  arXiv  Detail & Related papers  (2026-02-10T20:00:51Z)
• Learning Filter-Aware Distance Metrics for Nearest Neighbor Search with Multiple Filters [7.493563348124023]
  Filtered Approximate Nearest Neighbor (ANN) search retrieves the closest vectors for a query vector from a dataset.<n>Existing graph-based methods typically incorporate filter awareness by assigning fixed penalties.<n>We propose a principled alternative that learns the optimal trade-off between vector distance and filter match directly from the data.
  arXiv  Detail & Related papers  (2025-11-06T05:24:41Z)
• Prior-based Noisy Text Data Filtering: Fast and Strong Alternative For Perplexity [16.521507516831097]
  We propose a prior-based data filtering method that estimates token priors using corpus-level term frequency statistics.<n>Our approach filters documents based on the mean and standard deviation of token priors, serving as a fast proxy to PPL.<n>Despite its simplicity, the prior-based filter achieves the highest average performance across 20 downstream benchmarks.
  arXiv  Detail & Related papers  (2025-09-23T02:57:29Z)
• Attribute Filtering in Approximate Nearest Neighbor Search: An In-depth Experimental Study [18.5007917065799]
  We present a unified Filtering ANN search interface that encompasses the latest algorithms and evaluate them extensively from multiple perspectives.<n>First, we propose a comprehensive taxonomy of existing Filtering ANN algorithms based on attribute types and filtering strategies.<n>We then conduct a broad experimental evaluation on 10 algorithms and 12 methods across 4 datasets.
  arXiv  Detail & Related papers  (2025-08-22T09:54:57Z)
• MinPrompt: Graph-based Minimal Prompt Data Augmentation for Few-shot Question Answering [64.6741991162092]
  We present MinPrompt, a minimal data augmentation framework for open-domain question answering. We transform the raw text into a graph structure to build connections between different factual sentences. We then apply graph algorithms to identify the minimal set of sentences needed to cover the most information in the raw text. We generate QA pairs based on the identified sentence subset and train the model on the selected sentences to obtain the final model.
  arXiv  Detail & Related papers  (2023-10-08T04:44:36Z)
• JoinGym: An Efficient Query Optimization Environment for Reinforcement Learning [58.71541261221863]
  Join order selection (JOS) is the problem of ordering join operations to minimize total query execution cost. We present JoinGym, a query optimization environment for bushy reinforcement learning (RL) Under the hood, JoinGym simulates a query plan's cost by looking up intermediate result cardinalities from a pre-computed dataset.
  arXiv  Detail & Related papers  (2023-07-21T17:00:06Z)
• Data Selection for Language Models via Importance Resampling [90.9263039747723]
  We formalize the problem of selecting a subset of a large raw unlabeled dataset to match a desired target distribution. We extend the classic importance resampling approach used in low-dimensions for LM data selection. We instantiate the DSIR framework with hashed n-gram features for efficiency, enabling the selection of 100M documents in 4.5 hours.
  arXiv  Detail & Related papers  (2023-02-06T23:57:56Z)
• Learning Versatile Convolution Filters for Efficient Visual Recognition [125.34595948003745]
  This paper introduces versatile filters to construct efficient convolutional neural networks. We conduct theoretical analysis on network complexity and an efficient convolution scheme is introduced. Experimental results on benchmark datasets and neural networks demonstrate that our versatile filters are able to achieve comparable accuracy as that of original filters.
  arXiv  Detail & Related papers  (2021-09-20T06:07:14Z)
• Data Agnostic Filter Gating for Efficient Deep Networks [72.4615632234314]
  Current filter pruning methods mainly leverage feature maps to generate important scores for filters and prune those with smaller scores. In this paper, we propose a data filter pruning method that uses an auxiliary network named Dagger module to induce pruning. In addition, to help prune filters with certain FLOPs constraints, we leverage an explicit FLOPs-aware regularization to directly promote pruning filters toward target FLOPs.
  arXiv  Detail & Related papers  (2020-10-28T15:26:40Z)
• Dependency Aware Filter Pruning [74.69495455411987]
  Pruning a proportion of unimportant filters is an efficient way to mitigate the inference cost. Previous work prunes filters according to their weight norms or the corresponding batch-norm scaling factors. We propose a novel mechanism to dynamically control the sparsity-inducing regularization so as to achieve the desired sparsity.
  arXiv  Detail & Related papers  (2020-05-06T07:41:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.