Enhancing Long-Term Memory using Hierarchical Aggregate Tree for Retrieval Augmented Generation

• URL: http://arxiv.org/abs/2406.06124v1
• Date: Mon, 10 Jun 2024 09:29:08 GMT
• Title: Enhancing Long-Term Memory using Hierarchical Aggregate Tree for Retrieval Augmented Generation
• Authors: Aadharsh Aadhithya A, Sachin Kumar S, Soman K. P,
• Abstract summary: HAT encapsulates information from children nodes, enabling broad coverage with depth control. experiments show HAT improves dialog coherence and summary quality over baseline contexts.
• Score: 1.4665304971699265
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models have limited context capacity, hindering reasoning over long conversations. We propose the Hierarchical Aggregate Tree memory structure to recursively aggregate relevant dialogue context through conditional tree traversals. HAT encapsulates information from children nodes, enabling broad coverage with depth control. We formulate finding best context as optimal tree traversal. Experiments show HAT improves dialog coherence and summary quality over baseline contexts, demonstrating the techniques effectiveness for multi turn reasoning without exponential parameter growth. This memory augmentation enables more consistent, grounded longform conversations from LLMs

Related papers

• FragRel: Exploiting Fragment-level Relations in the External Memory of Large Language Models [54.13671100638092]
  We propose a fragment-connected Hierarchical Memory based Large Language Models (LLMs) We formulate the fragment-level relations in external memory and present several instantiations for different text types. We validate the benefits of involving these relations on long story understanding, repository-level code generation, and long-term chatting.
  arXiv  Detail & Related papers  (2024-06-05T09:31:37Z)
• Evaluating Very Long-Term Conversational Memory of LLM Agents [95.84027826745609]
  We introduce a machine-human pipeline to generate high-quality, very long-term dialogues. We equip each agent with the capability of sharing and reacting to images. The generated conversations are verified and edited by human annotators for long-range consistency.
  arXiv  Detail & Related papers  (2024-02-27T18:42:31Z)
• Walking Down the Memory Maze: Beyond Context Limit through Interactive Reading [63.93888816206071]
  We introduce MemWalker, a method that processes the long context into a tree of summary nodes. Upon receiving a query, the model navigates this tree in search of relevant information, and responds once it gathers sufficient information. We show that, beyond effective reading, MemWalker enhances explainability by highlighting the reasoning steps as it interactively reads the text; pinpointing the relevant text segments related to the query.
  arXiv  Detail & Related papers  (2023-10-08T06:18:14Z)
• Recursively Summarizing Enables Long-Term Dialogue Memory in Large Language Models [75.98775135321355]
  Given a long conversation, large language models (LLMs) fail to recall past information and tend to generate inconsistent responses. We propose to generate summaries/ memory using large language models (LLMs) to enhance long-term memory ability.
  arXiv  Detail & Related papers  (2023-08-29T04:59:53Z)
• TREA: Tree-Structure Reasoning Schema for Conversational Recommendation [23.29064805769382]
  We propose a novel Tree structure Reasoning schEmA named TREA. TREA constructs a multi-archhierical tree as the reasoning structure to clarify the causal relationships between mentioned entities. Extensive experiments on two public CRS datasets have demonstrated the effectiveness of our approach.
  arXiv  Detail & Related papers  (2023-07-20T02:48:04Z)
• Unsupervised Learning of Discourse Structures using a Tree Autoencoder [8.005512864082126]
  We propose a new strategy to generate tree structures in a task-agnostic, unsupervised fashion by extending a latent tree induction framework with an auto-encoding objective. The proposed approach can be applied to any tree objective, such as syntactic parsing, discourse parsing and others. In this paper we are inferring general tree structures of natural text in multiple domains, showing promising results on a diverse set of tasks.
  arXiv  Detail & Related papers  (2020-12-17T08:40:34Z)
• Reasoning in Dialog: Improving Response Generation by Context Reading Comprehension [49.92173751203827]
  In multi-turn dialog, utterances do not always take the full form of sentences. We propose to improve the response generation performance by examining the model's ability to answer a reading comprehension question.
  arXiv  Detail & Related papers  (2020-12-14T10:58:01Z)
• Rethinking Learnable Tree Filter for Generic Feature Transform [71.77463476808585]
  Learnable Tree Filter presents a remarkable approach to model structure-preserving relations for semantic segmentation. To relax the geometric constraint, we give the analysis by reformulating it as a Markov Random Field and introduce a learnable unary term. For semantic segmentation, we achieve leading performance (82.1% mIoU) on the Cityscapes benchmark without bells-and-whistles.
  arXiv  Detail & Related papers  (2020-12-07T07:16:47Z)
• MEGA RST Discourse Treebanks with Structure and Nuclearity from Scalable Distant Sentiment Supervision [30.615883375573432]
  We present a novel methodology to automatically generate discourse treebanks using distant supervision from sentiment-annotated datasets. Our approach generates trees incorporating structure and nuclearity for documents of arbitrary length by relying on an efficient beam-search strategy. Experiments indicate that a discourse trained on our MEGA-DT treebank delivers promising inter-domain performance gains.
  arXiv  Detail & Related papers  (2020-11-05T18:22:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.