WESE: Weak Exploration to Strong Exploitation for LLM Agents

• URL: http://arxiv.org/abs/2404.07456v1
• Date: Thu, 11 Apr 2024 03:31:54 GMT
• Title: WESE: Weak Exploration to Strong Exploitation for LLM Agents
• Authors: Xu Huang, Weiwen Liu, Xiaolong Chen, Xingmei Wang, Defu Lian, Yasheng Wang, Ruiming Tang, Enhong Chen,
• Abstract summary: This paper proposes a novel approach, Weak Exploration to Strong Exploitation (WESE) to enhance LLM agents in solving open-world interactive tasks. WESE involves decoupling the exploration and exploitation process, employing a cost-effective weak agent to perform exploration tasks for global knowledge. A knowledge graph-based strategy is then introduced to store the acquired knowledge and extract task-relevant knowledge, enhancing the stronger agent in success rate and efficiency for the exploitation task.
• Score: 95.6720931773781
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, large language models (LLMs) have demonstrated remarkable potential as an intelligent agent. However, existing researches mainly focus on enhancing the agent's reasoning or decision-making abilities through well-designed prompt engineering or task-specific fine-tuning, ignoring the procedure of exploration and exploitation. When addressing complex tasks within open-world interactive environments, these methods exhibit limitations. Firstly, the lack of global information of environments leads to greedy decisions, resulting in sub-optimal solutions. On the other hand, irrelevant information acquired from the environment not only adversely introduces noise, but also incurs additional cost. This paper proposes a novel approach, Weak Exploration to Strong Exploitation (WESE), to enhance LLM agents in solving open-world interactive tasks. Concretely, WESE involves decoupling the exploration and exploitation process, employing a cost-effective weak agent to perform exploration tasks for global knowledge. A knowledge graph-based strategy is then introduced to store the acquired knowledge and extract task-relevant knowledge, enhancing the stronger agent in success rate and efficiency for the exploitation task. Our approach is flexible enough to incorporate diverse tasks, and obtains significant improvements in both success rates and efficiency across four interactive benchmarks.

Related papers

• EfficientEQA: An Efficient Approach for Open Vocabulary Embodied Question Answering [21.114403949257934]
  Embodied Question Answering (EQA) is an essential yet challenging task for robotic home assistants. Recent studies have shown that large vision-language models (VLMs) can be effectively utilized for EQA, but existing works either focus on video-based question answering or rely on closed-form choice sets. We propose a novel framework called EfficientEQA for open-vocabulary EQA, which enables efficient exploration and accurate answering.
  arXiv  Detail & Related papers  (2024-10-26T19:48:47Z)
• EVOLvE: Evaluating and Optimizing LLMs For Exploration [76.66831821738927]
  Large language models (LLMs) remain under-studied in scenarios requiring optimal decision-making under uncertainty. We measure LLMs' (in)ability to make optimal decisions in bandits, a state-less reinforcement learning setting relevant to many applications. Motivated by the existence of optimal exploration algorithms, we propose efficient ways to integrate this algorithmic knowledge into LLMs.
  arXiv  Detail & Related papers  (2024-10-08T17:54:03Z)
• Synergistic Multi-Agent Framework with Trajectory Learning for Knowledge-Intensive Tasks [44.42989163847349]
  Large Language Models (LLMs) have led to significant breakthroughs in various natural language processing tasks. generating factually consistent responses in knowledge-intensive scenarios remains a challenge. This paper introduces SMART, a novel multi-agent framework that leverages external knowledge to enhance the interpretability and factual consistency of LLM-generated responses.
  arXiv  Detail & Related papers  (2024-07-13T13:58:24Z)
• DISCOVERYWORLD: A Virtual Environment for Developing and Evaluating Automated Scientific Discovery Agents [49.74065769505137]
  We introduce DISCOVERYWORLD, the first virtual environment for developing and benchmarking an agent's ability to perform complete cycles of novel scientific discovery. It includes 120 different challenge tasks spanning eight topics each with three levels of difficulty and several parametric variations. We find that strong baseline agents, that perform well in prior published environments, struggle on most DISCOVERYWORLD tasks.
  arXiv  Detail & Related papers  (2024-06-10T20:08:44Z)
• Collaborative Knowledge Infusion for Low-resource Stance Detection [83.88515573352795]
  Target-related knowledge is often needed to assist stance detection models. We propose a collaborative knowledge infusion approach for low-resource stance detection tasks.
  arXiv  Detail & Related papers  (2024-03-28T08:32:14Z)
• Mastering the Task of Open Information Extraction with Large Language Models and Consistent Reasoning Environment [52.592199835286394]
  Open Information Extraction (OIE) aims to extract objective structured knowledge from natural texts. Large language models (LLMs) have exhibited remarkable in-context learning capabilities.
  arXiv  Detail & Related papers  (2023-10-16T17:11:42Z)
• ExpeL: LLM Agents Are Experiential Learners [60.54312035818746]
  We introduce the Experiential Learning (ExpeL) agent to allow learning from agent experiences without requiring parametric updates. Our agent autonomously gathers experiences and extracts knowledge using natural language from a collection of training tasks. At inference, the agent recalls its extracted insights and past experiences to make informed decisions.
  arXiv  Detail & Related papers  (2023-08-20T03:03:34Z)
• Improving Knowledge Extraction from LLMs for Task Learning through Agent Analysis [4.055489363682198]
  Large language models (LLMs) offer significant promise as a knowledge source for task learning. Prompt engineering has been shown to be effective for eliciting knowledge from an LLM, but alone it is insufficient for acquiring relevant, situationally grounded knowledge for an embodied agent learning novel tasks. We describe a cognitive-agent approach, STARS, that extends and complements prompt engineering, mitigating its limitations and thus enabling an agent to acquire new task knowledge matched to its native language capabilities, embodiment, environment, and user preferences.
  arXiv  Detail & Related papers  (2023-06-11T20:50:14Z)
• Asking Before Acting: Gather Information in Embodied Decision Making with Language Models [20.282749796376063]
  We show that Large Language Models (LLMs) encounter challenges in efficiently gathering essential information in unfamiliar environments. We propose textitAsking Before Acting (ABA), a method that empowers the agent to proactively inquire with external sources for pertinent information using natural language. We conduct extensive experiments involving a spectrum of environments including text-based household everyday tasks, robot arm manipulation tasks, and real world open domain image based embodied tasks.
  arXiv  Detail & Related papers  (2023-05-25T04:05:08Z)
• KnowRU: Knowledge Reusing via Knowledge Distillation in Multi-agent Reinforcement Learning [16.167201058368303]
  Deep Reinforcement Learning (RL) algorithms have achieved dramatically progress in the multi-agent area. To alleviate this problem, efficient leveraging of the historical experience is essential. We propose a method, named "KnowRU" for knowledge reusing.
  arXiv  Detail & Related papers  (2021-03-27T12:38:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.