Leveraging Knowledge Graph-Based Human-Like Memory Systems to Solve Partially Observable Markov Decision Processes

• URL: http://arxiv.org/abs/2408.05861v2
• Date: Sun, 18 Aug 2024 19:32:35 GMT
• Title: Leveraging Knowledge Graph-Based Human-Like Memory Systems to Solve Partially Observable Markov Decision Processes
• Authors: Taewoon Kim, Vincent François-Lavet, Michael Cochez,
• Abstract summary: We have developed a partially observable Markov decision processes (POMDP) environment, where the agent has to answer questions while navigating a maze. The environment is completely knowledge graph (KG) based, where the hidden states are dynamic KGs. We train and compare agents with different memory systems, to shed light on how human brains work when it comes to managing its own memory.
• Score: 9.953497719634726
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Humans observe only part of their environment at any moment but can still make complex, long-term decisions thanks to our long-term memory. To test how an AI can learn and utilize its long-term memory, we have developed a partially observable Markov decision processes (POMDP) environment, where the agent has to answer questions while navigating a maze. The environment is completely knowledge graph (KG) based, where the hidden states are dynamic KGs. A KG is both human- and machine-readable, making it easy to see what the agents remember and forget. We train and compare agents with different memory systems, to shed light on how human brains work when it comes to managing its own memory. By repurposing the given learning objective as learning a memory management policy, we were able to capture the most likely hidden state, which is not only interpretable but also reusable.

Related papers

• Human-inspired Perspectives: A Survey on AI Long-term Memory [46.33545299110207]
  This paper introduces the mechanisms of human long-term memory, then explores AI long-term memory mechanisms. We propose the Cognitive Architecture of Self-Adaptive Long-term Memory (SALM) SALM provides a theoretical framework for the practice of AI long-term memory and holds potential for guiding the creation of next-generation long-term memory driven AI systems.
  arXiv  Detail & Related papers  (2024-11-01T10:04:01Z)
• Stable Hadamard Memory: Revitalizing Memory-Augmented Agents for Reinforcement Learning [64.93848182403116]
  Current deep-learning memory models struggle in reinforcement learning environments that are partially observable and long-term. We introduce the Stable Hadamard Memory, a novel memory model for reinforcement learning agents. Our approach significantly outperforms state-of-the-art memory-based methods on challenging partially observable benchmarks.
  arXiv  Detail & Related papers  (2024-10-14T03:50:17Z)
• Survey on Memory-Augmented Neural Networks: Cognitive Insights to AI Applications [4.9008611361629955]
  Memory-Augmented Neural Networks (MANNs) blend human-like memory processes into AI. The study investigates advanced architectures such as Hopfield Networks, Neural Turing Machines, Correlation Matrix Memories, Memformer, and Neural Attention Memory. It dives into real-world uses of MANNs across Natural Language Processing, Computer Vision, Multimodal Learning, and Retrieval Models.
  arXiv  Detail & Related papers  (2023-12-11T06:05:09Z)
• Semantic HELM: A Human-Readable Memory for Reinforcement Learning [9.746397419479445]
  We propose a novel memory mechanism that represents past events in human language. We train our memory mechanism on a set of partially observable environments and find that it excels on tasks that require a memory component. Since our memory mechanism is human-readable, we can peek at an agent's memory and check whether crucial pieces of information have been stored.
  arXiv  Detail & Related papers  (2023-06-15T17:47:31Z)
• LLM as A Robotic Brain: Unifying Egocentric Memory and Control [77.0899374628474]
  Embodied AI focuses on the study and development of intelligent systems that possess a physical or virtual embodiment (i.e. robots) Memory and control are the two essential parts of an embodied system and usually require separate frameworks to model each of them. We propose a novel framework called LLM-Brain: using Large-scale Language Model as a robotic brain to unify egocentric memory and control.
  arXiv  Detail & Related papers  (2023-04-19T00:08:48Z)
• Memory-Augmented Theory of Mind Network [59.9781556714202]
  Social reasoning requires the capacity of theory of mind (ToM) to contextualise and attribute mental states to others. Recent machine learning approaches to ToM have demonstrated that we can train the observer to read the past and present behaviours of other agents. We tackle the challenges by equipping the observer with novel neural memory mechanisms to encode, and hierarchical attention to selectively retrieve information about others. This results in ToMMY, a theory of mind model that learns to reason while making little assumptions about the underlying mental processes.
  arXiv  Detail & Related papers  (2023-01-17T14:48:58Z)
• A Machine with Short-Term, Episodic, and Semantic Memory Systems [9.42475956340287]
  Inspired by the cognitive science theory of the explicit human memory systems, we have modeled an agent with short-term, episodic, and semantic memory systems. Our experiments indicate that an agent with human-like memory systems can outperform an agent without this memory structure in the environment.
  arXiv  Detail & Related papers  (2022-12-05T08:34:23Z)
• Learning What to Memorize: Using Intrinsic Motivation to Form Useful Memory in Partially Observable Reinforcement Learning [0.0]
  In order to learn in an ambiguous environment, an agent has to keep previous perceptions in a memory. In this study, we follow the idea of giving the control of the memory to the agent by allowing it to have memory-changing actions. This learning mechanism is supported by an intrinsic motivation to memorize rare observations that can help the agent to disambiguate its state in the environment.
  arXiv  Detail & Related papers  (2021-10-25T11:15:54Z)
• Memory and attention in deep learning [19.70919701635945]
  Memory construction for machine is inevitable. Recent progresses on modeling memory in deep learning have revolved around external memory constructions. The aim of this thesis is to advance the understanding on memory and attention in deep learning.
  arXiv  Detail & Related papers  (2021-07-03T09:21:13Z)
• Learning to Learn Variational Semantic Memory [132.39737669936125]
  We introduce variational semantic memory into meta-learning to acquire long-term knowledge for few-shot learning. The semantic memory is grown from scratch and gradually consolidated by absorbing information from tasks it experiences. We formulate memory recall as the variational inference of a latent memory variable from addressed contents.
  arXiv  Detail & Related papers  (2020-10-20T15:05:26Z)
• Self-Attentive Associative Memory [69.40038844695917]
  We propose to separate the storage of individual experiences (item memory) and their occurring relationships (relational memory) We achieve competitive results with our proposed two-memory model in a diversity of machine learning tasks.
  arXiv  Detail & Related papers  (2020-02-10T03:27:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.