Reinforcement-learning robotic sailboats: simulator and preliminary
results
- URL: http://arxiv.org/abs/2402.03337v1
- Date: Tue, 16 Jan 2024 09:04:05 GMT
- Title: Reinforcement-learning robotic sailboats: simulator and preliminary
results
- Authors: Eduardo Charles Vasconcellos (UFF), Ronald M Sampaio, Andr\'e P D
Ara\'ujo (UFF), Esteban Walter Gonzales Clua, Philippe Preux (SEQUEL, GRAppA
- LIFL), Raphael Guerra, Luiz M G Gon\c{c}alves (UFRN), Luis Mart\'i, Hernan
Lira, Nayat Sanchez-Pi
- Abstract summary: This work focuses on the main challenges and problems in developing a virtual oceanic environment reproducing real experiments using Unmanned Surface Vehicles (USV) digital twins.
We introduce the key features for building virtual worlds, considering using Reinforcement Learning (RL) agents for autonomous navigation and control.
- Score: 0.37918614538294315
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: This work focuses on the main challenges and problems in developing a virtual
oceanic environment reproducing real experiments using Unmanned Surface
Vehicles (USV) digital twins. We introduce the key features for building
virtual worlds, considering using Reinforcement Learning (RL) agents for
autonomous navigation and control. With this in mind, the main problems concern
the definition of the simulation equations (physics and mathematics), their
effective implementation, and how to include strategies for simulated control
and perception (sensors) to be used with RL. We present the modeling,
implementation steps, and challenges required to create a functional digital
twin based on a real robotic sailing vessel. The application is immediate for
developing navigation algorithms based on RL to be applied on real boats.
Related papers
- Gaussian Splatting to Real World Flight Navigation Transfer with Liquid Networks [93.38375271826202]
We present a method to improve generalization and robustness to distribution shifts in sim-to-real visual quadrotor navigation tasks.
We first build a simulator by integrating Gaussian splatting with quadrotor flight dynamics, and then, train robust navigation policies using Liquid neural networks.
In this way, we obtain a full-stack imitation learning protocol that combines advances in 3D Gaussian splatting radiance field rendering, programming of expert demonstration training data, and the task understanding capabilities of Liquid networks.
arXiv Detail & Related papers (2024-06-21T13:48:37Z) - Aquatic Navigation: A Challenging Benchmark for Deep Reinforcement Learning [53.3760591018817]
We propose a new benchmarking environment for aquatic navigation using recent advances in the integration between game engines and Deep Reinforcement Learning.
Specifically, we focus on PPO, one of the most widely accepted algorithms, and we propose advanced training techniques.
Our empirical evaluation shows that a well-designed combination of these ingredients can achieve promising results.
arXiv Detail & Related papers (2024-05-30T23:20:23Z) - GarchingSim: An Autonomous Driving Simulator with Photorealistic Scenes
and Minimalist Workflow [24.789118651720045]
We introduce an autonomous driving simulator with photorealistic scenes.
The simulator is able to communicate with external algorithms through ROS2 or Socket.IO.
We implement a highly accurate vehicle dynamics model within the simulator to enhance the realism of the vehicle's physical effects.
arXiv Detail & Related papers (2024-01-28T23:26:15Z) - Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous
Driving Research [76.93956925360638]
Waymax is a new data-driven simulator for autonomous driving in multi-agent scenes.
It runs entirely on hardware accelerators such as TPUs/GPUs and supports in-graph simulation for training.
We benchmark a suite of popular imitation and reinforcement learning algorithms with ablation studies on different design decisions.
arXiv Detail & Related papers (2023-10-12T20:49:15Z) - Reinforcement Learning with Human Feedback for Realistic Traffic
Simulation [53.85002640149283]
Key element of effective simulation is the incorporation of realistic traffic models that align with human knowledge.
This study identifies two main challenges: capturing the nuances of human preferences on realism and the unification of diverse traffic simulation models.
arXiv Detail & Related papers (2023-09-01T19:29:53Z) - How Simulation Helps Autonomous Driving:A Survey of Sim2real, Digital
Twins, and Parallel Intelligence [16.24370001383615]
How to adapt driving knowledge learned in simulation to reality becomes a critical issue.
Virtual simulation world differs from the real world in many aspects such as lighting, textures, vehicle dynamics, and agents' behaviors.
Three categories of approaches to address the reality gap issue: transferring knowledge from simulation to reality (sim2real), learning in digital twins (DTs), and learning by parallel intelligence (PI)
arXiv Detail & Related papers (2023-05-02T09:00:32Z) - Residual Physics Learning and System Identification for Sim-to-real
Transfer of Policies on Buoyancy Assisted Legged Robots [14.760426243769308]
In this work, we demonstrate robust sim-to-real transfer of control policies on the BALLU robots via system identification.
Rather than relying on standard supervised learning formulations, we utilize deep reinforcement learning to train an external force policy.
We analyze the improved simulation fidelity by comparing the simulation trajectories against the real-world ones.
arXiv Detail & Related papers (2023-03-16T18:49:05Z) - Robust Reinforcement Learning-based Autonomous Driving Agent for
Simulation and Real World [0.0]
We present a DRL-based algorithm that is capable of performing autonomous robot control using Deep Q-Networks (DQN)
In our approach, the agent is trained in a simulated environment and it is able to navigate both in a simulated and real-world environment.
The trained agent is able to run on limited hardware resources and its performance is comparable to state-of-the-art approaches.
arXiv Detail & Related papers (2020-09-23T15:23:54Z) - Point Cloud Based Reinforcement Learning for Sim-to-Real and Partial
Observability in Visual Navigation [62.22058066456076]
Reinforcement Learning (RL) represents powerful tools to solve complex robotic tasks.
RL does not work directly in the real-world, which is known as the sim-to-real transfer problem.
We propose a method that learns on an observation space constructed by point clouds and environment randomization.
arXiv Detail & Related papers (2020-07-27T17:46:59Z) - RoboTHOR: An Open Simulation-to-Real Embodied AI Platform [56.50243383294621]
We introduce RoboTHOR to democratize research in interactive and embodied visual AI.
We show there exists a significant gap between the performance of models trained in simulation when they are tested in both simulations and their carefully constructed physical analogs.
arXiv Detail & Related papers (2020-04-14T20:52:49Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.