Memory-Augmented Architecture for Long-Term Context Handling in Large Language Models

• URL: http://arxiv.org/abs/2506.18271v1
• Date: Mon, 23 Jun 2025 03:57:25 GMT
• Title: Memory-Augmented Architecture for Long-Term Context Handling in Large Language Models
• Authors: Haseeb Ullah Khan Shinwari, Muhammad Usama,
• Abstract summary: Large Language Models face challenges in maintaining coherent interactions over extended dialogues due to their limited contextual memory.<n>We propose a memory-augmented architecture that dynamically retrieves, updates, and prunes relevant information from past interactions, ensuring effective long-term context handling.
• Score: 0.46040036610482665
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models face significant challenges in maintaining coherent interactions over extended dialogues due to their limited contextual memory. This limitation often leads to fragmented exchanges and reduced relevance in responses, diminishing user experience. To address these issues, we propose a memory-augmented architecture that dynamically retrieves, updates, and prunes relevant information from past interactions, ensuring effective long-term context handling. Experimental results demonstrate that our solution significantly improves contextual coherence, reduces memory overhead, and enhances response quality, showcasing its potential for real-time applications in interactive systems.

Related papers

• UniConv: Unifying Retrieval and Response Generation for Large Language Models in Conversations [71.79210031338464]
  We show how to unify dense retrieval and response generation for large language models in conversation.<n>We conduct joint fine-tuning with different objectives and design two mechanisms to reduce the inconsistency risks.<n>The evaluations on five conversational search datasets demonstrate that our unified model can mutually improve both tasks and outperform the existing baselines.
  arXiv  Detail & Related papers  (2025-07-09T17:02:40Z)
• FindingDory: A Benchmark to Evaluate Memory in Embodied Agents [49.89792845476579]
  We introduce a new benchmark for long-range embodied tasks in the Habitat simulator.<n>This benchmark evaluates memory-based capabilities across 60 tasks requiring sustained engagement and contextual awareness.
  arXiv  Detail & Related papers  (2025-06-18T17:06:28Z)
• Structured Memory Mechanisms for Stable Context Representation in Large Language Models [16.929937978584917]
  The model integrates explicit memory units, gated writing mechanisms, and attention-based reading modules.<n>A forgetting function is introduced to enable dynamic updates of memory content.<n>The model achieves clear advantages in text generation consistency, stability in multi-turn question answering, and accuracy in cross-context reasoning.
  arXiv  Detail & Related papers  (2025-05-28T22:49:04Z)
• Towards Multi-Granularity Memory Association and Selection for Long-Term Conversational Agents [73.77930932005354]
  We propose MemGAS, a framework that enhances memory consolidation by constructing multi-granularity association, adaptive selection, and retrieval.<n>MemGAS is based on multi-granularity memory units and employs Gaussian Mixture Models to cluster and associate new memories with historical ones.<n>Experiments on four long-term memory benchmarks demonstrate that MemGAS outperforms state-of-the-art methods on both question answer and retrieval tasks.
  arXiv  Detail & Related papers  (2025-05-26T06:13:07Z)
• In Prospect and Retrospect: Reflective Memory Management for Long-term Personalized Dialogue Agents [70.12342024019044]
  Large Language Models (LLMs) have made significant progress in open-ended dialogue, yet their inability to retain and retrieve relevant information limits their effectiveness.<n>We propose Reflective Memory Management (RMM), a novel mechanism for long-term dialogue agents, integrating forward- and backward-looking reflections.<n>RMM shows more than 10% accuracy improvement over the baseline without memory management on the LongMemEval dataset.
  arXiv  Detail & Related papers  (2025-03-11T04:15:52Z)
• Contextual Memory Reweaving in Large Language Models Using Layered Latent State Reconstruction [0.0]
  Token dependencies degrade as sequence length increases, leading to a decline in coherence and factual consistency.<n>A structured approach is introduced to mitigate this issue through the reweaving of latent states captured at different processing layers.<n>The proposed Contextual Memory Reweaving framework incorporates a Layered Latent State Reconstruction mechanism.
  arXiv  Detail & Related papers  (2025-02-04T06:25:20Z)
• QRMeM: Unleash the Length Limitation through Question then Reflection Memory Mechanism [46.441032033076034]
  Memory mechanism offers a flexible solution for managing long contexts. We introduce a novel strategy, Question then Reflection Memory Mechanism (QRMeM), incorporating a dual-structured memory pool. Our evaluation across multiple-choice questions (MCQ) and multi-document question answering (Multi-doc QA) benchmarks showcases QRMeM enhanced performance compared to existing approaches.
  arXiv  Detail & Related papers  (2024-06-19T02:46:18Z)
• Towards Lifelong Dialogue Agents via Timeline-based Memory Management [26.95907827895548]
  We present THEANINE, a framework for lifelong dialogue agents.<n> THEANINE discards memory removal and manages large-scale memories by linking them based on their temporal and cause-effect relation.<n>Along with THEANINE, we introduce TeaFarm, a counterfactual-driven evaluation scheme.
  arXiv  Detail & Related papers  (2024-06-16T16:17:46Z)
• Ever-Evolving Memory by Blending and Refining the Past [30.63352929849842]
  CREEM is a novel memory system for long-term conversation. It seamlessly connects past and present information, while also possessing the ability to forget obstructive information.
  arXiv  Detail & Related papers  (2024-03-03T08:12:59Z)
• Evaluating Very Long-Term Conversational Memory of LLM Agents [95.84027826745609]
  We introduce a machine-human pipeline to generate high-quality, very long-term dialogues. We equip each agent with the capability of sharing and reacting to images. The generated conversations are verified and edited by human annotators for long-range consistency.
  arXiv  Detail & Related papers  (2024-02-27T18:42:31Z)
• Re-mine, Learn and Reason: Exploring the Cross-modal Semantic Correlations for Language-guided HOI detection [57.13665112065285]
  Human-Object Interaction (HOI) detection is a challenging computer vision task. We present a framework that enhances HOI detection by incorporating structured text knowledge.
  arXiv  Detail & Related papers  (2023-07-25T14:20:52Z)
• Ranking Enhanced Dialogue Generation [77.8321855074999]
  How to effectively utilize the dialogue history is a crucial problem in multi-turn dialogue generation. Previous works usually employ various neural network architectures to model the history. This paper proposes a Ranking Enhanced Dialogue generation framework.
  arXiv  Detail & Related papers  (2020-08-13T01:49:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.