BeSimulator: A Large Language Model Powered Text-based Behavior Simulator

• URL: http://arxiv.org/abs/2409.15865v1
• Date: Tue, 24 Sep 2024 08:37:04 GMT
• Title: BeSimulator: A Large Language Model Powered Text-based Behavior Simulator
• Authors: Jianan Wang, Bin Li, Xueying Wang, Fu Li, Yunlong Wu, Juan Chen, Xiaodong Yi,
• Abstract summary: We introduce BeSimulator as an attempt towards behavior simulation in the context of text-based environments. BeSimulator can generalize across scenarios and achieve long-horizon complex simulation.
• Score: 28.112491177744783
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Traditional robot simulators focus on physical process modeling and realistic rendering, often suffering from high computational costs, inefficiencies, and limited adaptability. To handle this issue, we propose Behavior Simulation in robotics to emphasize checking the behavior logic of robots and achieving sufficient alignment between the outcome of robot actions and real scenarios. In this paper, we introduce BeSimulator, a modular and novel LLM-powered framework, as an attempt towards behavior simulation in the context of text-based environments. By constructing text-based virtual environments and performing semantic-level simulation, BeSimulator can generalize across scenarios and achieve long-horizon complex simulation. Inspired by human cognition processes, it employs a "consider-decide-capture-transfer" methodology, termed Chain of Behavior Simulation, which excels at analyzing action feasibility and state transitions. Additionally, BeSimulator incorporates code-driven reasoning to enable arithmetic operations and enhance reliability, as well as integrates reflective feedback to refine simulation. Based on our manually constructed behavior-tree-based simulation benchmark BTSIMBENCH, our experiments show a significant performance improvement in behavior simulation compared to baselines, ranging from 14.7% to 26.6%.

Related papers

• GenSim: A General Social Simulation Platform with Large Language Model based Agents [111.00666003559324]
  We propose a novel large language model (LLMs)-based simulation platform called textitGenSim. Our platform supports one hundred thousand agents to better simulate large-scale populations in real-world contexts. To our knowledge, GenSim represents an initial step toward a general, large-scale, and correctable social simulation platform.
  arXiv  Detail & Related papers  (2024-10-06T05:02:23Z)
• FactorSim: Generative Simulation via Factorized Representation [14.849320460718591]
  We introduce FACTORSIM that generates full simulations in code from language input that can be used to train agents. For evaluation, we introduce a generative simulation benchmark that assesses the generated simulation code's accuracy and effectiveness in facilitating zero-shot transfers in reinforcement learning settings. We show that FACTORSIM outperforms existing methods in generating simulations regarding prompt alignment (e.g., accuracy), zero-shot transfer abilities, and human evaluation.
  arXiv  Detail & Related papers  (2024-09-26T09:00:30Z)
• EAGERx: Graph-Based Framework for Sim2real Robot Learning [9.145895178276822]
  Sim2real, that is, the transfer of learned control policies from simulation to real world, is an area of growing interest in robotics. We introduce EAGERx, a framework with a unified software pipeline for both real and simulated robot learning.
  arXiv  Detail & Related papers  (2024-07-05T08:01:19Z)
• DrEureka: Language Model Guided Sim-To-Real Transfer [64.14314476811806]
  Transferring policies learned in simulation to the real world is a promising strategy for acquiring robot skills at scale. In this paper, we investigate using Large Language Models (LLMs) to automate and accelerate sim-to-real design. Our approach is capable of solving novel robot tasks, such as quadruped balancing and walking atop a yoga ball.
  arXiv  Detail & Related papers  (2024-06-04T04:53:05Z)
• Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research [76.93956925360638]
  Waymax is a new data-driven simulator for autonomous driving in multi-agent scenes. It runs entirely on hardware accelerators such as TPUs/GPUs and supports in-graph simulation for training. We benchmark a suite of popular imitation and reinforcement learning algorithms with ablation studies on different design decisions.
  arXiv  Detail & Related papers  (2023-10-12T20:49:15Z)
• Near-realtime Facial Animation by Deep 3D Simulation Super-Resolution [7.14576106770047]
  We present a neural network-based simulation framework that can efficiently and realistically enhance a facial performance produced by a low-cost, realtime physics-based simulation. We use face animation as an exemplar of such a simulation domain, where creating this semantic congruence is achieved by simply dialing in the same muscle actuation controls and skeletal pose in the two simulators. Our proposed neural network super-resolution framework generalizes from this training set to unseen expressions, compensates for modeling discrepancies between the two simulations due to limited resolution or cost-cutting approximations in the real-time variant, and does not require any semantic descriptors or parameters to
  arXiv  Detail & Related papers  (2023-05-05T00:09:24Z)
• Analyzing and Enhancing Closed-loop Stability in Reactive Simulation [25.27603440925488]
  We propose a new reactive simulation framework to bridge the human behavior gap between simulation and real-world traffic scenarios. We first propose a new reactive simulation framework, where the smoothness and consistency of the simulated state sequences are crucial factors to stability. We then incorporate the kinematic vehicle model into the framework to improve the closed-loop stability of the reactive simulation.
  arXiv  Detail & Related papers  (2022-08-09T06:31:35Z)
• Robot Learning from Randomized Simulations: A Review [59.992761565399185]
  Deep learning has caused a paradigm shift in robotics research, favoring methods that require large amounts of data. State-of-the-art approaches learn in simulation where data generation is fast as well as inexpensive. We focus on a technique named 'domain randomization' which is a method for learning from randomized simulations.
  arXiv  Detail & Related papers  (2021-11-01T13:55:41Z)
• SimNet: Computer Architecture Simulation using Machine Learning [3.7019798164954336]
  This work describes a concerted effort, where machine learning (ML) is used to accelerate discrete-event simulation. A GPU-accelerated parallel simulator is implemented based on the proposed instruction latency predictor. Its simulation accuracy and throughput are validated and evaluated against a state-of-the-art simulator.
  arXiv  Detail & Related papers  (2021-05-12T17:31:52Z)
• A User's Guide to Calibrating Robotics Simulators [54.85241102329546]
  This paper proposes a set of benchmarks and a framework for the study of various algorithms aimed to transfer models and policies learnt in simulation to the real world. We conduct experiments on a wide range of well known simulated environments to characterize and offer insights into the performance of different algorithms. Our analysis can be useful for practitioners working in this area and can help make informed choices about the behavior and main properties of sim-to-real algorithms.
  arXiv  Detail & Related papers  (2020-11-17T22:24:26Z)
• Point Cloud Based Reinforcement Learning for Sim-to-Real and Partial Observability in Visual Navigation [62.22058066456076]
  Reinforcement Learning (RL) represents powerful tools to solve complex robotic tasks. RL does not work directly in the real-world, which is known as the sim-to-real transfer problem. We propose a method that learns on an observation space constructed by point clouds and environment randomization.
  arXiv  Detail & Related papers  (2020-07-27T17:46:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.