Deep reinforcement learning for tracking a moving target in jellyfish-like swimming

• URL: http://arxiv.org/abs/2409.08815v1
• Date: Fri, 13 Sep 2024 13:29:46 GMT
• Title: Deep reinforcement learning for tracking a moving target in jellyfish-like swimming
• Authors: Yihao Chen, Yue Yang,
• Abstract summary: We develop a deep reinforcement learning method for training a jellyfish-like swimmer to track a moving target in a two-dimensional flow. We employ a deep Q-network (DQN) that takes the swimmer's geometry and dynamic parameters as inputs, and outputs actions which are the forces applied to the swimmer.
• Score: 16.727471388063893
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We develop a deep reinforcement learning method for training a jellyfish-like swimmer to effectively track a moving target in a two-dimensional flow. This swimmer is a flexible object equipped with a muscle model based on torsional springs. We employ a deep Q-network (DQN) that takes the swimmer's geometry and dynamic parameters as inputs, and outputs actions which are the forces applied to the swimmer. In particular, we introduce an action regulation to mitigate the interference from complex fluid-structure interactions. The goal of these actions is to navigate the swimmer to a target point in the shortest possible time. In the DQN training, the data on the swimmer's motions are obtained from simulations conducted using the immersed boundary method. During tracking a moving target, there is an inherent delay between the application of forces and the corresponding response of the swimmer's body due to hydrodynamic interactions between the shedding vortices and the swimmer's own locomotion. Our tests demonstrate that the swimmer, with the DQN agent and action regulation, is able to dynamically adjust its course based on its instantaneous state. This work extends the application scope of machine learning in controlling flexible objects within fluid environments.

Related papers

• Spatial-Temporal Graph Diffusion Policy with Kinematic Modeling for Bimanual Robotic Manipulation [88.83749146867665]
  Existing approaches learn a policy to predict a distant next-best end-effector pose. They then compute the corresponding joint rotation angles for motion using inverse kinematics. We propose Kinematics enhanced Spatial-TemporAl gRaph diffuser.
  arXiv  Detail & Related papers  (2025-03-13T17:48:35Z)
• Fine Tuning Swimming Locomotion Learned from Mosquito Larvae [0.9349784561232036]
  In prior research, we analyzed the backwards swimming motion of mosquito larvae, parameterized it, and replicated it in a Computational Fluid Dynamics (CFD) model. In this project, we further optimize this copied solution for the model of the swimmer.
  arXiv  Detail & Related papers  (2024-11-16T06:54:43Z)
• Optimal-state Dynamics Estimation for Physics-based Human Motion Capture from Videos [6.093379844890164]
  We propose a novel method to selectively incorporate the physics models with the kinematics observations in an online setting. A recurrent neural network is introduced to realize a Kalman filter that attentively balances the kinematics input and simulated motion. The proposed approach excels in the physics-based human pose estimation task and demonstrates the physical plausibility of the predictive dynamics.
  arXiv  Detail & Related papers  (2024-10-10T10:24:59Z)
• FAFA: Frequency-Aware Flow-Aided Self-Supervision for Underwater Object Pose Estimation [65.01601309903971]
  We introduce FAFA, a Frequency-Aware Flow-Aided self-supervised framework for 6D pose estimation of unmanned underwater vehicles (UUVs) Our framework relies solely on the 3D model and RGB images, alleviating the need for any real pose annotations or other-modality data like depths. We evaluate the effectiveness of FAFA on common underwater object pose benchmarks and showcase significant performance improvements compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-09-25T03:54:01Z)
• Modular Neural Network Policies for Learning In-Flight Object Catching with a Robot Hand-Arm System [55.94648383147838]
  We present a modular framework designed to enable a robot hand-arm system to learn how to catch flying objects. Our framework consists of five core modules: (i) an object state estimator that learns object trajectory prediction, (ii) a catching pose quality network that learns to score and rank object poses for catching, (iii) a reaching control policy trained to move the robot hand to pre-catch poses, and (iv) a grasping control policy trained to perform soft catching motions. We conduct extensive evaluations of our framework in simulation for each module and the integrated system, to demonstrate high success rates of in-flight
  arXiv  Detail & Related papers  (2023-12-21T16:20:12Z)
• Learning to swim efficiently in a nonuniform flow field [0.0]
  Microswimmers can acquire information on the surrounding fluid by sensing mechanical queues. We study how local and non-local information can be used to train a swimmer to achieve particular swimming tasks.
  arXiv  Detail & Related papers  (2022-12-22T04:51:47Z)
• Fast Aquatic Swimmer Optimization with Differentiable Projective Dynamics and Neural Network Hydrodynamic Models [23.480913364381664]
  Aquatic locomotion is a classic fluid-structure interaction (FSI) problem of interest to biologists and engineers. We present a novel, fully differentiable hybrid approach to FSI that combines a 2D numerical simulation for the deformable solid structure of the swimmer. We demonstrate the computational efficiency and differentiability of our hybrid simulator on a 2D carangiform swimmer.
  arXiv  Detail & Related papers  (2022-03-30T15:21:44Z)
• Benchmarking Safe Deep Reinforcement Learning in Aquatic Navigation [78.17108227614928]
  We propose a benchmark environment for Safe Reinforcement Learning focusing on aquatic navigation. We consider a value-based and policy-gradient Deep Reinforcement Learning (DRL) We also propose a verification strategy that checks the behavior of the trained models over a set of desired properties.
  arXiv  Detail & Related papers  (2021-12-16T16:53:56Z)
• OSCAR: Data-Driven Operational Space Control for Adaptive and Robust Robot Manipulation [50.59541802645156]
  Operational Space Control (OSC) has been used as an effective task-space controller for manipulation. We propose OSC for Adaptation and Robustness (OSCAR), a data-driven variant of OSC that compensates for modeling errors. We evaluate our method on a variety of simulated manipulation problems, and find substantial improvements over an array of controller baselines.
  arXiv  Detail & Related papers  (2021-10-02T01:21:38Z)
• Learning swimming escape patterns under energy constraints [6.014777261874645]
  Flow simulations have identified escape patterns consistent with those observed in natural larval swimmers. We deploy reinforcement learning to discover swimmer escape patterns under energy constraints.
  arXiv  Detail & Related papers  (2021-05-03T11:58:37Z)
• Movement Tracks for the Automatic Detection of Fish Behavior in Videos [63.85815474157357]
  We offer a dataset of sablefish (Anoplopoma fimbria) startle behaviors in underwater videos, and investigate the use of deep learning (DL) methods for behavior detection on it. Our proposed detection system identifies fish instances using DL-based frameworks, determines trajectory tracks, derives novel behavior-specific features, and employs Long Short-Term Memory (LSTM) networks to identify startle behavior in sablefish.
  arXiv  Detail & Related papers  (2020-11-28T05:51:19Z)
• Learning to swim in potential flow [17.146927368452598]
  We propose a simple model of a three-link fish swimming in a potential flow environment. We arrive at optimal shape changes for two swimming tasks. Although the fish has no direct control over the drift itself, it learns to take advantage of the presence of moderate drift to reach its target.
  arXiv  Detail & Related papers  (2020-09-30T06:31:27Z)
• ReLMoGen: Leveraging Motion Generation in Reinforcement Learning for Mobile Manipulation [99.2543521972137]
  ReLMoGen is a framework that combines a learned policy to predict subgoals and a motion generator to plan and execute the motion needed to reach these subgoals. Our method is benchmarked on a diverse set of seven robotics tasks in photo-realistic simulation environments. ReLMoGen shows outstanding transferability between different motion generators at test time, indicating a great potential to transfer to real robots.
  arXiv  Detail & Related papers  (2020-08-18T08:05:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.