LLM-POET: Evolving Complex Environments using Large Language Models

• URL: http://arxiv.org/abs/2406.04663v1
• Date: Fri, 7 Jun 2024 06:23:07 GMT
• Title: LLM-POET: Evolving Complex Environments using Large Language Models
• Authors: Fuma Aki, Riku Ikeda, Takumi Saito, Ciaran Regan, Mizuki Oka,
• Abstract summary: We propose LLM-POET, a modification of the POET algorithm where the environment is both created and mutated using a Large Language Model (LLM) We found that not only could the LLM produce a diverse range of environments, but compared to the CPPNs used in Enhanced-POET for environment generation, the LLM allowed for a 34% increase in the performance gain of co-evolution.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Creating systems capable of generating virtually infinite variations of complex and novel behaviour without predetermined goals or limits is a major challenge in the field of AI. This challenge has been addressed through the development of several open-ended algorithms that can continuously generate new and diverse behaviours, such as the POET and Enhanced-POET algorithms for co-evolving environments and agent behaviour. One of the challenges with existing methods however, is that they struggle to continuously generate complex environments. In this work, we propose LLM-POET, a modification of the POET algorithm where the environment is both created and mutated using a Large Language Model (LLM). By fine-tuning a LLM with text representations of Evolution Gym environments and captions that describe the environment, we were able to generate complex and diverse environments using natural language. We found that not only could the LLM produce a diverse range of environments, but compared to the CPPNs used in Enhanced-POET for environment generation, the LLM allowed for a 34% increase in the performance gain of co-evolution. This increased performance suggests that the agents were able to learn a more diverse set of skills by training on more complex environments.

Related papers

• Scaling Autonomous Agents via Automatic Reward Modeling And Planning [52.39395405893965]
  Large language models (LLMs) have demonstrated remarkable capabilities across a range of tasks. However, they still struggle with problems requiring multi-step decision-making and environmental feedback. We propose a framework that can automatically learn a reward model from the environment without human annotations.
  arXiv  Detail & Related papers  (2025-02-17T18:49:25Z)
• EnvBridge: Bridging Diverse Environments with Cross-Environment Knowledge Transfer for Embodied AI [7.040779338576156]
  Large Language Models (LLMs) can generate text planning or control code for robots. These methods still face challenges in terms of flexibility and applicability across different environments. We propose EnvBridge to enhance the adaptability and robustness of robotic manipulation agents.
  arXiv  Detail & Related papers  (2024-10-22T11:52:22Z)
• AgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generation [89.68433168477227]
  Large Language Model (LLM) based agents have garnered significant attention and are becoming increasingly popular. This paper investigates enhancing the planning abilities of LLMs through instruction tuning. To address this limitation, this paper explores the automated synthesis of diverse environments and a gradual range of planning tasks.
  arXiv  Detail & Related papers  (2024-08-01T17:59:46Z)
• Exploring the Improvement of Evolutionary Computation via Large Language Models [3.4641800438055297]
  Evolutionary computation (EC) has been applied across various domains. As the complexity of problems increases, the limitations of EC have become more apparent. By harnessing large language models' vast knowledge and adaptive capabilities, we provide a forward-looking overview of potential improvements.
  arXiv  Detail & Related papers  (2024-05-05T10:13:55Z)
• Can large language models explore in-context? [87.49311128190143]
  We deploy Large Language Models as agents in simple multi-armed bandit environments. We find that the models do not robustly engage in exploration without substantial interventions.
  arXiv  Detail & Related papers  (2024-03-22T17:50:43Z)
• Entropy-Regularized Token-Level Policy Optimization for Language Agent Reinforcement [67.1393112206885]
  Large Language Models (LLMs) have shown promise as intelligent agents in interactive decision-making tasks. We introduce Entropy-Regularized Token-level Policy Optimization (ETPO), an entropy-augmented RL method tailored for optimizing LLMs at the token level. We assess the effectiveness of ETPO within a simulated environment that models data science code generation as a series of multi-step interactive tasks.
  arXiv  Detail & Related papers  (2024-02-09T07:45:26Z)
• When large language models meet evolutionary algorithms [48.213640761641926]
  Pre-trained large language models (LLMs) have powerful capabilities for generating creative natural text. Evolutionary algorithms (EAs) can discover diverse solutions to complex real-world problems. Motivated by the common collective and directionality of text generation and evolution, this paper illustrates the parallels between LLMs and EAs.
  arXiv  Detail & Related papers  (2024-01-19T05:58:30Z)
• Connecting Large Language Models with Evolutionary Algorithms Yields Powerful Prompt Optimizers [70.18534453485849]
  EvoPrompt is a framework for discrete prompt optimization. It borrows the idea of evolutionary algorithms (EAs) as they exhibit good performance and fast convergence. It significantly outperforms human-engineered prompts and existing methods for automatic prompt generation.
  arXiv  Detail & Related papers  (2023-09-15T16:50:09Z)
• Cooperative coevolutionary Modified Differential Evolution with Distance-based Selection for Large-Scale Optimization Problems in noisy environments through an automatic Random Grouping [3.274290296343038]
  We propose an automatic Random Grouping (aRG) to solve large-scale optimization problems in noisy environments. We also introduce Modified Evolution with Distance-based Selection (MDE-DS) to enhance the ability in noisy environments. Our proposal has broad prospects to solve LSOPs in noisy environments and can be easily extended to higher-dimensional problems.
  arXiv  Detail & Related papers  (2022-09-02T01:37:17Z)
• Deep Surrogate Assisted Generation of Environments [7.217405582720078]
  Quality diversity (QD) optimization has been proven to be an effective component of environment generation algorithms. We propose Deep Surrogate Assisted Generation of Environments (DSAGE), a sample-efficient QD environment generation algorithm. Results in two benchmark domains show that DSAGE significantly outperforms existing QD environment generation algorithms.
  arXiv  Detail & Related papers  (2022-06-09T00:14:03Z)
• Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design [121.73425076217471]
  We propose Unsupervised Environment Design (UED), where developers provide environments with unknown parameters, and these parameters are used to automatically produce a distribution over valid, solvable environments. We call our technique Protagonist Antagonist Induced Regret Environment Design (PAIRED) Our experiments demonstrate that PAIRED produces a natural curriculum of increasingly complex environments, and PAIRED agents achieve higher zero-shot transfer performance when tested in highly novel environments.
  arXiv  Detail & Related papers  (2020-12-03T17:37:01Z)
• Lifelong Incremental Reinforcement Learning with Online Bayesian Inference [11.076005074172516]
  A long-lived reinforcement learning agent is to incrementally adapt its behavior as its environment changes. We propose LifeLong Reinforcement Learning (LLIRL), a new incremental algorithm for efficient lifelong adaptation to dynamic environments.
  arXiv  Detail & Related papers  (2020-07-28T13:23:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.