OrchANN: A Unified I/O Orchestration Framework for Skewed Out-of-Core Vector Search

• URL: http://arxiv.org/abs/2512.22838v1
• Date: Sun, 28 Dec 2025 08:42:38 GMT
• Title: OrchANN: A Unified I/O Orchestration Framework for Skewed Out-of-Core Vector Search
• Abstract summary: Approximate nearest neighbor search (ANNS) at billion scale is fundamentally an out-of-core problem.<n>We present OrchANN, an out-of-core ANNS engine that uses an I/O orchestration model for unified I/O governance.
• Score: 38.2797436367431
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Approximate nearest neighbor search (ANNS) at billion scale is fundamentally an out-of-core problem: vectors and indexes live on SSD, so performance is dominated by I/O rather than compute. Under skewed semantic embeddings, existing out-of-core systems break down: a uniform local index mismatches cluster scales, static routing misguides queries and inflates the number of probed partitions, and pruning is incomplete at the cluster level and lossy at the vector level, triggering "fetch-to-discard" reranking on raw vectors. We present OrchANN, an out-of-core ANNS engine that uses an I/O orchestration model for unified I/O governance along the route-access-verify pipeline. OrchANN selects a heterogeneous local index per cluster via offline auto-profiling, maintains a query-aware in-memory navigation graph that adapts to skewed workloads, and applies multi-level pruning with geometric bounds to filter both clusters and vectors before issuing SSD reads. Across five standard datasets under strict out-of-core constraints, OrchANN outperforms four baselines including DiskANN, Starling, SPANN, and PipeANN in both QPS and latency while reducing SSD accesses. Furthermore, OrchANN delivers up to 17.2x higher QPS and 25.0x lower latency than competing systems without sacrificing accuracy.

Related papers

• NeuRoute: Logit-Guided Neural Routing for Billion-Scale Vector Search with Sub-Hour Index Construction [6.022622976208937]
  NeuRoute is a learned hashing index that turns short binary codes into an effective routing primitive for large-scale vector search.<n>NeuRoute trains a lightweight neural network encoder with a selective similarity-preserving objective to produce well-balanced binary addresses.
  arXiv  Detail & Related papers  (2026-08-15T22:44:10Z)
• You Only Index Once: Cross-Layer Sparse Attention with Shared Routing [61.29627714699688]
  Cross-layer sparse attention (A) is built on top of KV-sharing architectures such as YOCO.<n>A single indexer computes token-level top-k selection once and reuses the resulting index across layers.<n>Results show CLSA is both accurate and efficient, achieving up to 7.6x decoding speedup and 17.1x overall throughput improvement at 128K context.
  arXiv  Detail & Related papers  (2026-06-04T17:54:04Z)
• Lodestar: An Online-Learning LLM Inference Router [6.819928135489163]
  Large language model (LLM) inference tasks are crucial for user-perceived latency.<n>Lodstar is a novel learning-based request routing system for distributed GPU clusters.<n>Lodstar achieves 1.41x lower average TTFT and 1.47x lower P99 TTFT on average.
  arXiv  Detail & Related papers  (2026-05-31T01:31:02Z)
• HISA: Efficient Hierarchical Indexing for Fine-Grained Sparse Attention [62.79085204939384]
  HISA (Hierarchical Indexed Sparse Attention) rewrites the search path from a flat token scan into a two-stage hierarchical procedure.<n>On kernel-level benchmarks, HISA achieves up to speedup at 64K context.
  arXiv  Detail & Related papers  (2026-03-30T13:59:51Z)
• IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reuse [68.18308357205586]
  Longcontext agentic have emerged as a defining use case for large language models.<n>Sparse attention addresses this challenge effectively, and DeepSeek Sparse Attention (DSA) is a representative production-grade solution.<n>We present IndexCache, which exploits cross-layer redundancy by partitioning layers into a small set of Full layers that run their own indexers and a majority of Shared layers that simply reuse the nearest Full layer's top-k indices.
  arXiv  Detail & Related papers  (2026-03-12T17:27:21Z)
• ECHO: Encoding Communities via High-order Operators [8.970269049715933]
  Community detection in attributed networks faces a fundamental divide: topological algorithms ignore semantic features, while Graph Neural Networks (GNNs) encounter devastating computational bottlenecks.<n>We introduce ECHO, a scalable, self supervised architecture that reframes community detection as an adaptive, multi scale diffusion process.<n>ECHO completely bypasses traditional O(N2) memory bottlenecks without sacrificing the mathematical precision of global gradients.
  arXiv  Detail & Related papers  (2026-02-25T22:14:29Z)
• DiEC: Diffusion Embedded Clustering [0.76629754443761]
  Deep clustering critically depends on representations that expose clear cluster structure.<n>Most prior methods learn a single embedding with an autoencoder or a self-supervised encoder and treat it as the primary representation for clustering.<n>We propose Embedded Diffusion Clustering (DiEC), an unsupervised clustering framework that exploits this trajectory by directly leveraging intermediate activations of a pretrained diffusion U-Net.
  arXiv  Detail & Related papers  (2025-12-24T03:10:00Z)
• B+ANN: A Fast Billion-Scale Disk-based Nearest-Neighbor Index [3.4720326275852]
  We present a novel disk-based ANN index, B+ANN, to address issues of the HNSW algorithm.<n>It partitions input data into blocks containing semantically similar items, then builds an B+ tree variant to store blocks both in-memory and on disks.<n>It improves both quality (Recall value), and execution performance (Queries per second/QPS) over HNSW.
  arXiv  Detail & Related papers  (2025-11-19T15:50:28Z)
• Scalable Disk-Based Approximate Nearest Neighbor Search with Page-Aligned Graph [3.994346326254537]
  We propose PageANN, a disk-based approximate nearest neighbor search (ANNS) framework for high performance and scalability.<n>Results show that PageANN significantly outperforms state-of-the-art (SOTA) disk-based ANNS methods, achieving 1.85x-10.83x higher throughput and 51.7%-91.9% lower latency across different datasets and memory budgets.
  arXiv  Detail & Related papers  (2025-09-29T20:44:13Z)
• DISTRIBUTEDANN: Efficient Scaling of a Single DISKANN Graph Across Thousands of Computers [9.854925314375024]
  We present DISTRIBUTEDANN, a distributed vector search service that makes it possible to search over a single 50 billion vector graph index spread across over a thousand machines.<n>This is 6x more efficient than existing partitioning and routing strategies that route the vector query to a subset of partitions in a scale out vector search system.
  arXiv  Detail & Related papers  (2025-09-07T13:13:02Z)
• Divide-Then-Rule: A Cluster-Driven Hierarchical Interpolator for Attribute-Missing Graphs [51.13363550716544]
  Deep graph clustering is an unsupervised task aimed at partitioning nodes with incomplete attributes into distinct clusters.<n>Existing imputation methods for attribute-missing graphs often fail to account for the varying amounts of information available across node neighborhoods.<n>We propose Divide-Then-Rule Graph Completion (DTRGC) to address this issue.
  arXiv  Detail & Related papers  (2025-07-12T03:33:19Z)
• HAKES: Scalable Vector Database for Embedding Search Service [16.034584281180006]
  We build a vector database that achieves high throughput and high recall under concurrent read-write workloads.<n>Our index outperforms index baselines in the high recall region and under concurrent read-write workloads.<n>namesys is scalable and achieves up to $16times$ higher throughputs than the baselines.
  arXiv  Detail & Related papers  (2025-05-18T19:26:29Z)
• FusionLLM: A Decentralized LLM Training System on Geo-distributed GPUs with Adaptive Compression [55.992528247880685]
  Decentralized training faces significant challenges regarding system design and efficiency. We present FusionLLM, a decentralized training system designed and implemented for training large deep neural networks (DNNs) We show that our system and method can achieve 1.45 - 9.39x speedup compared to baseline methods while ensuring convergence.
  arXiv  Detail & Related papers  (2024-10-16T16:13:19Z)
• Timely Asynchronous Hierarchical Federated Learning: Age of Convergence [59.96266198512243]
  We consider an asynchronous hierarchical federated learning setting with a client-edge-cloud framework. The clients exchange the trained parameters with their corresponding edge servers, which update the locally aggregated model. The goal of each client is to converge to the global model, while maintaining timeliness of the clients.
  arXiv  Detail & Related papers  (2023-06-21T17:39:16Z)
• Dink-Net: Neural Clustering on Large Graphs [59.10189693120368]
  A deep graph clustering method (Dink-Net) is proposed with the idea of dilation and shrink. By discriminating nodes, whether being corrupted by augmentations, representations are learned in a self-supervised manner. The clustering distribution is optimized by minimizing the proposed cluster dilation loss and cluster shrink loss. Compared to the runner-up, Dink-Net 9.62% achieves NMI improvement on the ogbn-papers100M dataset with 111 million nodes and 1.6 billion edges.
  arXiv  Detail & Related papers  (2023-05-28T15:33:24Z)
• Towards Deeper Graph Neural Networks with Differentiable Group Normalization [61.20639338417576]
  Graph neural networks (GNNs) learn the representation of a node by aggregating its neighbors. Over-smoothing is one of the key issues which limit the performance of GNNs as the number of layers increases. We introduce two over-smoothing metrics and a novel technique, i.e., differentiable group normalization (DGN)
  arXiv  Detail & Related papers  (2020-06-12T07:18:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.