DySK-Attn: A Framework for Efficient, Real-Time Knowledge Updating in Large Language Models via Dynamic Sparse Knowledge Attention

• URL: http://arxiv.org/abs/2508.07185v1
• Date: Sun, 10 Aug 2025 05:22:38 GMT
• Title: DySK-Attn: A Framework for Efficient, Real-Time Knowledge Updating in Large Language Models via Dynamic Sparse Knowledge Attention
• Authors: Kabir Khan, Priya Sharma, Arjun Mehta, Neha Gupta, Ravi Narayanan,
• Abstract summary: Large Language Models (LLMs) suffer from a critical limitation: their knowledge is static and quickly becomes outdated.<n>We propose DySK-Attn, a novel framework that enables LLMs to efficiently integrate real-time knowledge from a dynamic external source.
• Score: 2.168994673914057
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) suffer from a critical limitation: their knowledge is static and quickly becomes outdated. Retraining these massive models is computationally prohibitive, while existing knowledge editing techniques can be slow and may introduce unforeseen side effects. To address this, we propose DySK-Attn, a novel framework that enables LLMs to efficiently integrate real-time knowledge from a dynamic external source. Our approach synergizes an LLM with a dynamic Knowledge Graph (KG) that can be updated instantaneously. The core of our framework is a sparse knowledge attention mechanism, which allows the LLM to perform a coarse-to-fine grained search, efficiently identifying and focusing on a small, highly relevant subset of facts from the vast KG. This mechanism avoids the high computational cost of dense attention over the entire knowledge base and mitigates noise from irrelevant information. We demonstrate through extensive experiments on time-sensitive question-answering tasks that DySK-Attn significantly outperforms strong baselines, including standard Retrieval-Augmented Generation (RAG) and model editing techniques, in both factual accuracy for updated knowledge and computational efficiency. Our framework offers a scalable and effective solution for building LLMs that can stay current with the ever-changing world.

Related papers

• Massive Editing for Large Language Models Based on Dynamic Weight Generation [51.34392079812964]
  This paper proposes a Massive editing approach for Large Language Models (LLMs) based on dynamic weight Generation (MeG)<n>Our MeG can significantly improve the performance of large-scale knowledge editing in terms of Reliability, Generality, and Locality metrics.
  arXiv  Detail & Related papers  (2025-12-16T13:32:55Z)
• Improving Factuality in LLMs via Inference-Time Knowledge Graph Construction [30.381290814338413]
  Large Language Models (LLMs) often struggle with producing factually consistent answers due to limitations in their parametric memory.<n>We propose a novel framework that dynamically constructs and expands knowledge graphs (KGs) during inference.
  arXiv  Detail & Related papers  (2025-08-31T16:36:40Z)
• Continuously Steering LLMs Sensitivity to Contextual Knowledge with Proxy Models [33.022457807599075]
  CSKS (Continuously Steering Knowledge Sensitivity) is a framework that steers large language models' sensitivity to contextual knowledge continuously at a lightweight cost.<n>We show that our framework achieves continuous and precise control over LLMs' sensitivity to contextual knowledge, enabling both increased sensitivity and reduced sensitivity.
  arXiv  Detail & Related papers  (2025-08-27T09:30:24Z)
• TRAIL: Joint Inference and Refinement of Knowledge Graphs with Large Language Models [5.678291291711662]
  TRAIL is a novel, unified framework for Thinking, Reasoning, And Incremental Learning.<n>It couples joint inference and dynamic KG refinement with large language models.<n>Extensive experiments on multiple benchmarks demonstrate that TRAIL outperforms existing KG-augmented and retrieval-augmented LLM baselines by 3% to 13%.
  arXiv  Detail & Related papers  (2025-08-06T14:25:05Z)
• Efficient Tuning of Large Language Models for Knowledge-Grounded Dialogue Generation [21.52726424882653]
  We introduce KEDiT, an efficient method for fine-tuning large language models for knowledge-grounded dialogue generation.<n> KEDiT operates in two main phases: first, it employs an information bottleneck to compress retrieved knowledge into learnable parameters, retaining essential information while minimizing computational overhead.<n> experimental results on the Wizard of Wikipedia and a newly constructed PubMed-Dialog dataset demonstrate that KEDiT excels in generating contextually relevant and informative responses.
  arXiv  Detail & Related papers  (2025-04-10T13:54:36Z)
• KBM: Delineating Knowledge Boundary for Adaptive Retrieval in Large Language Models [69.99274367773997]
  Large Language Models (LLMs) often struggle with dynamically changing knowledge and handling unknown static information.<n>Retrieval-Augmented Generation (RAG) is employed to tackle these challenges and has a significant impact on improving LLM performance.<n>We propose a Knowledge Boundary Model (KBM) to express the known/unknown of a given question, and to determine whether a RAG needs to be triggered.
  arXiv  Detail & Related papers  (2024-11-09T15:12:28Z)
• Bridging LLMs and KGs without Fine-Tuning: Intermediate Probing Meets Subgraph-Aware Entity Descriptions [49.36683223327633]
  Large Language Models (LLMs) encapsulate extensive world knowledge and exhibit powerful context modeling capabilities.<n>We propose a novel framework that synergizes the strengths of LLMs with robust knowledge representation to enable effective and efficient KGC.<n>We achieve a 47% relative improvement over previous methods based on non-fine-tuned LLMs and, to our knowledge, are the first to achieve classification performance comparable to fine-tuned LLMs.
  arXiv  Detail & Related papers  (2024-08-13T10:15:55Z)
• Large Language Models-guided Dynamic Adaptation for Temporal Knowledge Graph Reasoning [87.10396098919013]
  Large Language Models (LLMs) have demonstrated extensive knowledge and remarkable proficiency in temporal reasoning.<n>We propose a Large Language Models-guided Dynamic Adaptation (LLM-DA) method for reasoning on Temporal Knowledge Graphs.<n>LLM-DA harnesses the capabilities of LLMs to analyze historical data and extract temporal logical rules.
  arXiv  Detail & Related papers  (2024-05-23T04:54:37Z)
• A Comprehensive Study of Knowledge Editing for Large Language Models [82.65729336401027]
  Large Language Models (LLMs) have shown extraordinary capabilities in understanding and generating text that closely mirrors human communication. This paper defines the knowledge editing problem and provides a comprehensive review of cutting-edge approaches. We introduce a new benchmark, KnowEdit, for a comprehensive empirical evaluation of representative knowledge editing approaches.
  arXiv  Detail & Related papers  (2024-01-02T16:54:58Z)
• Thrust: Adaptively Propels Large Language Models with External Knowledge [69.50273822565363]
  Large-scale pre-trained language models (PTLMs) are shown to encode rich knowledge in their model parameters.<n>The inherent knowledge in PTLMs can be opaque or static, making external knowledge necessary.<n>We propose the instance-level adaptive propulsion of external knowledge (IAPEK), where we only conduct the retrieval when necessary.
  arXiv  Detail & Related papers  (2023-07-19T20:16:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.