From Static to Dynamic: A Streaming RAG Approach to Real-time Knowledge Base

• URL: http://arxiv.org/abs/2508.05662v1
• Date: Thu, 31 Jul 2025 14:03:19 GMT
• Title: From Static to Dynamic: A Streaming RAG Approach to Real-time Knowledge Base
• Abstract summary: Streaming RAG is a unified pipeline that combines cosine screening, mini-batch clustering, and a heavy-hitter filter to maintain a compact prototype set.<n>Experiments on eight real-time streams show statistically significant gains in Recall@10 (up to 3 points, p 0.01), end-to-end latency below 15 ms, and throughput above 900 documents per second under a 150 MB budget.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dynamic streams from news feeds, social media, sensor networks, and financial markets challenge static RAG frameworks. Full-scale indices incur high memory costs; periodic rebuilds introduce latency that undermines data freshness; naive sampling sacrifices semantic coverage. We present Streaming RAG, a unified pipeline that combines multi-vector cosine screening, mini-batch clustering, and a counter-based heavy-hitter filter to maintain a compact prototype set. We further prove an approximation bound \$E\[R(K\_t)] \ge R^\* - L \Delta\$ linking retrieval quality to clustering variance. An incremental index upsert mechanism refreshes prototypes without interrupting queries. Experiments on eight real-time streams show statistically significant gains in Recall\@10 (up to 3 points, p < 0.01), end-to-end latency below 15 ms, and throughput above 900 documents per second under a 150 MB budget. Hyperparameter sensitivity analysis over cluster count, admission probability, relevance threshold, and counter capacity validates default settings. In open-domain question answering with GPT-3.5 Turbo, we record 3.2-point gain in Exact Match and 2.8-point gain in F1 on SQuAD; abstractive summarization yields ROUGE-L improvements. Streaming RAG establishes a new Pareto frontier for retrieval augmentation.

Related papers

• CateKV: On Sequential Consistency for Long-Context LLM Inference Acceleration [67.17709877327232]
  Cate KV is a hybrid KV cache method that retains only critical token information for consistent heads.<n>Cate KV reduces memory usage by up to $2.72times$ and accelerates decoding by $2.18times$ in single-sample inputs.
  arXiv  Detail & Related papers  (2026-08-31T06:02:37Z)
• TransRetrieval: Scaling Up Transformer-Based Retrieval for Industrial Recommendation [19.45139517853295]
  We present TransRetrieval, a Transformer-based retrieval framework that scales with both computational budget and cross-domain data.<n>On a 40-billion-interaction industrial dataset, scaling compute from 0.1 to 2 MFLOPs per target yields +19.3/+22.2 pt Recall@2000.<n>In online A/B tests, TransRetrieval lifts platform revenue by 2.53% under the same end-to-end latency constraint as the production baseline.
  arXiv  Detail & Related papers  (2026-08-26T08:35:16Z)
• Adaptive Re-Ranking [9.210588331263503]
  We propose Adaptive Re-Ranking, a utility-based labeling framework for cost-aware routing.<n>We train a routing classifier with 3 strategies: sparse retrieval (BM25), dense re-ranking (MiniLM-L6-v2), and heavy neural re-ranking (BGE-v2-m3)<n>Compared to BGE our method achieves 1.15-53x lower median latency and 1.11-5.22x lower mean latency across all datasets we have tested, while delivering -17.5% to +4.0% nDCG@10.
  arXiv  Detail & Related papers  (2026-06-24T00:12:44Z)
• MARQUIS: A Three-Stage Pipeline for Video Retrieval-Augmented Generation [64.33427079838602]
  Retrieval-augmented generation from videos requires systems to retrieve relevant audiovisual evidence from large corpora and synthesize it into coherent, attributed text.<n>We present MARQUIS: a three-stage pipeline that addresses these limitations through query expansion, fusion, and reranking.<n>For article generation, ITER-QA-BASE improves average human score from 3.09 to 3.83 over the CAG baseline, while MARQUIS-RLM achieves a human score of 3.30 and the strongest citation recall among non-QA systems.
  arXiv  Detail & Related papers  (2026-05-17T20:19:04Z)
• When to Retrieve During Reasoning: Adaptive Retrieval for Large Reasoning Models [13.891522069967507]
  We introduce ReaLM-Retrieve, a reasoning-aware retrieval framework.<n>Step-level uncertainty detector identifies knowledge gaps at reasoning-step rather than token or sentence level.<n>Retrieval intervention policy learns when external evidence maximally benefits ongoing reasoning.
  arXiv  Detail & Related papers  (2026-04-29T13:15:44Z)
• Accelerating Quantum Tensor Network Simulations with Unified Path Variations and Non-Degenerate Batched Sampling [8.539489108260872]
  Pre-Trajectory Sampling with Batched Execution (PTSBE) has dramatically increased the data collection rate of these methods.<n>While PTSBE has demonstrated data collection speedups of over $106 times$, tensor network implementations only achieved $sim 15 times$ speedup.<n>We develop 1) error-independent unified path variation, 2) non-degenerate tensor network sampling, and 3) a flexible/optimized contraction framework.
  arXiv  Detail & Related papers  (2026-04-09T17:02:18Z)
• Randomization Boosts KV Caching, Learning Balances Query Load: A Joint Perspective [31.67506313325633]
  KV caching is a technique for accelerating Large Language Model (LLM) inference by reusing key-value ( KV) pairs from previous queries.<n>The default Least Recently Used (LRU) eviction algorithm struggles with dynamic online query arrivals.<n>We give the first unified mathematical model that captures the core trade-offs between KV cache eviction and query routing.
  arXiv  Detail & Related papers  (2026-01-26T22:20:59Z)
• SimpleMem: Efficient Lifelong Memory for LLM Agents [73.74399447715052]
  We introduce SimpleMem, an efficient memory framework based on semantic lossless compression.<n>We propose a three-stage pipeline designed to maximize information density and token utilization.<n> Experiments on benchmark datasets show that our method consistently outperforms baseline approaches in accuracy, retrieval efficiency, and inference cost.
  arXiv  Detail & Related papers  (2026-01-05T21:02:49Z)
• PUFM++: Point Cloud Upsampling via Enhanced Flow Matching [15.738247394527024]
  PUFM++ is an enhanced flow-matching framework for reconstructing point clouds from sparse, noisy, and partial observations.<n>We introduce a two-stage flow-matching strategy that first learns a direct, straight-path flow from sparse inputs to dense targets, and then refines it using noise-perturbed samples to approximate the terminal marginal distribution better.<n>Experiments on synthetic benchmarks and real-world scans show that PUFM++ sets a new state of the art in point cloud upsampling.
  arXiv  Detail & Related papers  (2025-12-24T06:30:42Z)
• RaCoT: Plug-and-Play Contrastive Example Generation Mechanism for Enhanced LLM Reasoning Reliability [12.67288560758937]
  We propose RaCoT (Retrieval-aware Contrastive-of-Thought), a novel framework that shifts contrastive thinking to the pre-retrieval stage.<n>RaCoT guides the model to proactively focus on the critical details that determine answer divergence"
  arXiv  Detail & Related papers  (2025-10-26T15:06:44Z)
• vAttention: Verified Sparse Attention [100.98210818821688]
  vAttention is a practical sparse attention mechanism with user-specified $(epsilon, delta)$ guarantees on approximation accuracy (thus, verified)<n>We show that vAttention significantly improves the quality of sparse attention across datasets.<n>It can be deployed in reasoning scenarios to achieve fast decoding without compromising model quality.
  arXiv  Detail & Related papers  (2025-10-07T08:46:08Z)
• Exploring Efficient Open-Vocabulary Segmentation in the Remote Sensing [55.291219073365546]
  Open-Vocabulary Remote Sensing Image (OVRSIS) is an emerging task that adapts Open-Vocabulary (OVS) to the remote sensing (RS) domain.<n>textbfRSKT-Seg is a novel open-vocabulary segmentation framework tailored for remote sensing.<n> RSKT-Seg consistently outperforms strong OVS baselines by +3.8 mIoU and +5.9 mACC, while achieving 2x faster inference through efficient aggregation.
  arXiv  Detail & Related papers  (2025-09-15T15:24:49Z)
• PINGS: Physics-Informed Neural Network for Fast Generative Sampling [0.0]
  We introduce PINGS, a framework that amortizes diffusion sampling by training a physics-informed network to approximate reverse-time probability-flow dynamics.<n>As a proof of concept, we learn a direct map from a 3D standard normal to a non-Gaussian Gaussian Mixture Model (GMM)<n>PINGS frames generative sampling as a PINN-style residual problem with endpoint anchoring, yielding a white-box, differentiable map with NFE = 1.
  arXiv  Detail & Related papers  (2025-09-14T14:22:33Z)
• PentaRAG: Large-Scale Intelligent Knowledge Retrieval for Enterprise LLM Applications [5.4838799162708245]
  We present PentaRAG, a five-layer module that routes each query through two instant caches.<n>We show that PentaRAG cuts average GPU time to 0.248 seconds per query, roughly half that of a naive RAG baseline.<n>Results demonstrate that a layered routing strategy can deliver freshness, speed, and efficiency simultaneously in production-grade RAG systems.
  arXiv  Detail & Related papers  (2025-06-18T07:54:53Z)
• SkipVAR: Accelerating Visual Autoregressive Modeling via Adaptive Frequency-Aware Skipping [30.85025293160079]
  High-frequency components, or later steps, in the generation process contribute disproportionately to inference latency.<n>We identify two primary sources of inefficiency: step redundancy and unconditional branch redundancy.<n>We propose an automatic step-skipping strategy that selectively omits unnecessary generation steps to improve efficiency.
  arXiv  Detail & Related papers  (2025-06-10T15:35:29Z)
• vCache: Verified Semantic Prompt Caching [75.87215136638828]
  This paper proposes vCache, the first verified semantic cache with user-defined error rate guarantees.<n>It employs an online learning algorithm to estimate an optimal threshold for each cached prompt, enabling reliable cache responses without additional training.<n>Our experiments show that vCache consistently meets the specified error bounds while outperforming state-of-the-art static-threshold and fine-tuned embedding baselines.
  arXiv  Detail & Related papers  (2025-02-06T04:16:20Z)
• DASA: Delay-Adaptive Multi-Agent Stochastic Approximation [64.32538247395627]
  We consider a setting in which $N$ agents aim to speedup a common Approximation problem by acting in parallel and communicating with a central server. To mitigate the effect of delays and stragglers, we propose textttDASA, a Delay-Adaptive algorithm for multi-agent Approximation.
  arXiv  Detail & Related papers  (2024-03-25T22:49:56Z)
• HiRE: High Recall Approximate Top-$k$ Estimation for Efficient LLM Inference [68.59839755875252]
  HiRE comprises of two novel components: (i) a compression scheme to cheaply predict top-$k$ rows/columns with high recall, followed by full computation restricted to the predicted subset, and (ii) DA-TOP-$k$: an efficient multi-device approximate top-$k$ operator. We demonstrate that on a one billion parameter model, HiRE applied to both the softmax as well as feedforward layers, achieves almost matching pretraining and downstream accuracy, and speeds up inference latency by $1.47times$ on a single TPUv5e device.
  arXiv  Detail & Related papers  (2024-02-14T18:04:36Z)
• ATM Fraud Detection using Streaming Data Analytics [3.4543720783285052]
  In the study, we proposed ATM fraud detection in static and streaming contexts respectively. In both contexts, RF turned out to be the best model. RF is also empirically proven to be statistically significant than the next-best performing models.
  arXiv  Detail & Related papers  (2023-03-08T23:40:18Z)
• Accelerating Training and Inference of Graph Neural Networks with Fast Sampling and Pipelining [58.10436813430554]
  Mini-batch training of graph neural networks (GNNs) requires a lot of computation and data movement. We argue in favor of performing mini-batch training with neighborhood sampling in a distributed multi-GPU environment. We present a sequence of improvements to mitigate these bottlenecks, including a performance-engineered neighborhood sampler. We also conduct an empirical analysis that supports the use of sampling for inference, showing that test accuracies are not materially compromised.
  arXiv  Detail & Related papers  (2021-10-16T02:41:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.