Walk Wisely on Graph: Knowledge Graph Reasoning with Dual Agents via Efficient Guidance-Exploration

• URL: http://arxiv.org/abs/2408.01880v1
• Date: Sat, 3 Aug 2024 23:15:57 GMT
• Title: Walk Wisely on Graph: Knowledge Graph Reasoning with Dual Agents via Efficient Guidance-Exploration
• Authors: Zijian Wang, Bin Wang, Haifeng Jing, Huayu Li, Hongbo Dou,
• Abstract summary: We propose a multi-hop reasoning model with dual agents based on hierarchical reinforcement learning (HRL) FULORA tackles the above reasoning challenges by eFficient GUidance-ExpLORAtion between dual agents. Experiments conducted on three real-word knowledge graph datasets demonstrate that FULORA outperforms RL-based baselines.
• Score: 6.137115941053124
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent years, multi-hop reasoning has been widely studied for knowledge graph (KG) reasoning due to its efficacy and interpretability. However, previous multi-hop reasoning approaches are subject to two primary shortcomings. First, agents struggle to learn effective and robust policies at the early phase due to sparse rewards. Second, these approaches often falter on specific datasets like sparse knowledge graphs, where agents are required to traverse lengthy reasoning paths. To address these problems, we propose a multi-hop reasoning model with dual agents based on hierarchical reinforcement learning (HRL), which is named FULORA. FULORA tackles the above reasoning challenges by eFficient GUidance-ExpLORAtion between dual agents. The high-level agent walks on the simplified knowledge graph to provide stage-wise hints for the low-level agent walking on the original knowledge graph. In this framework, the low-level agent optimizes a value function that balances two objectives: (1) maximizing return, and (2) integrating efficient guidance from the high-level agent. Experiments conducted on three real-word knowledge graph datasets demonstrate that FULORA outperforms RL-based baselines, especially in the case of long-distance reasoning.

Related papers

• From Novice to Expert: LLM Agent Policy Optimization via Step-wise Reinforcement Learning [62.54484062185869]
  We introduce StepAgent, which utilizes step-wise reward to optimize the agent's reinforcement learning process. We propose implicit-reward and inverse reinforcement learning techniques to facilitate agent reflection and policy adjustment.
  arXiv  Detail & Related papers  (2024-11-06T10:35:11Z)
• DARA: Decomposition-Alignment-Reasoning Autonomous Language Agent for Question Answering over Knowledge Graphs [70.54226917774933]
  We propose the DecompositionAlignment-Reasoning Agent (DARA) framework. DARA effectively parses questions into formal queries through a dual mechanism. We show that DARA attains performance comparable to state-of-the-art enumerating-and-ranking-based methods for KGQA.
  arXiv  Detail & Related papers  (2024-06-11T09:09:37Z)
• Inferring Latent Temporal Sparse Coordination Graph for Multi-Agent Reinforcement Learning [19.386588137176933]
  We propose a Latent Temporal Sparse Coordination Graph (LTS-CG) for Multi-Agent Reinforcement Learning (MARL) This graph learning process is further augmented by two innovative characteristics: Predict-Future, which enables agents to foresee upcoming observations, and Infer-Present, ensuring a thorough grasp of the environmental context from limited data. Our demonstrated results on the StarCraft II benchmark underscore LTS-CG's superior performance.
  arXiv  Detail & Related papers  (2024-03-28T09:20:15Z)
• Bigraph Matching Weighted with Learnt Incentive Function for Multi-Robot Task Allocation [5.248564173595024]
  This paper develops a Graph Reinforcement Learning framework to learn the robustnesss or incentives for a bipartite graph matching approach to Multi-Robot Task Allocation. The performance of this new bigraph matching approach augmented with a GRL-derived incentive is found to be at par with the original bigraph matching approach.
  arXiv  Detail & Related papers  (2024-03-11T19:55:08Z)
• KnowAgent: Knowledge-Augmented Planning for LLM-Based Agents [54.09074527006576]
  Large Language Models (LLMs) have demonstrated great potential in complex reasoning tasks, yet they fall short when tackling more sophisticated challenges. This inadequacy primarily stems from the lack of built-in action knowledge in language agents. We introduce KnowAgent, a novel approach designed to enhance the planning capabilities of LLMs by incorporating explicit action knowledge.
  arXiv  Detail & Related papers  (2024-03-05T16:39:12Z)
• Ladder-of-Thought: Using Knowledge as Steps to Elevate Stance Detection [73.31406286956535]
  We introduce the Ladder-of-Thought (LoT) for the stance detection task. LoT directs the small LMs to assimilate high-quality external knowledge, refining the intermediate rationales produced. Our empirical evaluations underscore LoT's efficacy, marking a 16% improvement over GPT-3.5 and a 10% enhancement compared to GPT-3.5 with CoT on stance detection task.
  arXiv  Detail & Related papers  (2023-08-31T14:31:48Z)
• RACCER: Towards Reachable and Certain Counterfactual Explanations for Reinforcement Learning [2.0341936392563063]
  We propose RACCER, the first-specific approach to generating counterfactual explanations for the behavior of RL agents. We use a tree search to find the most suitable counterfactuals based on the defined properties. We evaluate RACCER in two tasks as well as conduct a user study to show that RL-specific counterfactuals help users better understand agents' behavior.
  arXiv  Detail & Related papers  (2023-03-08T09:47:00Z)
• Retrieval-Augmented Reinforcement Learning [63.32076191982944]
  We train a network to map a dataset of past experiences to optimal behavior. The retrieval process is trained to retrieve information from the dataset that may be useful in the current context. We show that retrieval-augmented R2D2 learns significantly faster than the baseline R2D2 agent and achieves higher scores.
  arXiv  Detail & Related papers  (2022-02-17T02:44:05Z)
• Learning to Walk with Dual Agents for Knowledge Graph Reasoning [20.232810842082674]
  Multi-hop reasoning approaches only work well on short reasoning paths and tend to miss the target entity with the increasing path length. We propose a dual-agent reinforcement learning framework, which trains two agents (GIANT and DWARF) to walk over a KG jointly and search for the answer collaboratively. Our approach tackles the reasoning challenge in long paths by assigning one of the agents (GIANT) searching on cluster-level paths quickly and providing stage-wise hints for another agent (DWARF)
  arXiv  Detail & Related papers  (2021-12-23T23:03:24Z)
• Agent-Centric Representations for Multi-Agent Reinforcement Learning [12.577354830985012]
  We investigate whether object-centric representations are also beneficial in the fully cooperative multi-agent reinforcement learning setting. Specifically, we study two ways of incorporating an agent-centric inductive bias into our RL algorithm. We evaluate these approaches on the Google Research Football environment as well as DeepMind Lab 2D.
  arXiv  Detail & Related papers  (2021-04-19T15:43:40Z)
• Explainability in Deep Reinforcement Learning [68.8204255655161]
  We review recent works in the direction to attain Explainable Reinforcement Learning (XRL) In critical situations where it is essential to justify and explain the agent's behaviour, better explainability and interpretability of RL models could help gain scientific insight on the inner workings of what is still considered a black box.
  arXiv  Detail & Related papers  (2020-08-15T10:11:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.