Reinforcement Learning Enables Autonomous Microrobot Navigation and Intervention in Simulated Blood Capillaries

• URL: http://arxiv.org/abs/2606.26154v1
• Date: Tue, 23 Jun 2026 12:21:18 GMT
• Title: Reinforcement Learning Enables Autonomous Microrobot Navigation and Intervention in Simulated Blood Capillaries
• Abstract summary: We develop a physically grounded simulation of a blood capillary network.<n>We train deep reinforcement learning agents to navigate it via chemotaxis.<n>These results establish RL as a viable framework for developing autonomous microrobotic intervention strategies in complex biological environments.
• Score: 0.903934726553365
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Autonomous microrobots navigating biological vasculature could enable targeted drug delivery and thrombolysis, yet training control policies for realistic environments remains an open challenge. Prior reinforcement learning (RL) studies of microrobotic navigation have been limited to idealized geometries that omit complex hydrodynamic flow fields, confined branching structures, and dense cellular obstacles found in vivo. Here, we develop a physically grounded simulation of a blood capillary network, incorporating realistic hydrodynamic flow fields, explicit red blood cell dynamics, and anatomically derived branching geometry, and train deep RL agents to navigate it via chemotaxis. We systematically map the physical limits of navigation across robot size and swimming speed, revealing a forbidden regime where Brownian motion and flow overcome propulsion. Successful agents independently discover multiple universal strategy types, including run-and-rotate and energy-efficient search-and-sit policies, regardless of robot parameters. Without retraining, these agents perform targeted blocking and unblocking of capillary flow, restoring throughput to healthy baseline levels. These results establish RL as a viable framework for developing autonomous microrobotic intervention strategies in complex biological environments.

Related papers

• Physics-Informed Modeling and Control of Emergent Behaviors in Robot Swarms [3.6125056660720425]
  PhyS is a physics-informed micro-macro-warm framework that represents multi-stage swarm emergence.<n>In several proof-of-concept swarm missions, we demonstrate that PhyS can generate distinct multi-stage emergent behaviors.<n>These results establish a physics-informed route for learning, interpreting and controlling emergent behaviors in robot swarms.
  arXiv  Detail & Related papers  (2026-06-01T02:50:45Z)
• Micro-Swarm Locomotion Optimization in Dynamic Flow using Multi-Objective Multi-Agent Reinforcement Learning [2.0223202343873226]
  Coordinating micro-robotic swarms in physiologically realistic, time-dependent fluid environments remains an unsolved challenge.<n>We present a hybrid Computational Fluid Dynamics - Multi-Objective Multi-Agent Reinforcement Learning framework.<n>16 magnetically actuated micro-robots navigate a pulsatile arterial, simultaneously optimizing upstream progression, energy conservation, and motion smoothness.
  arXiv  Detail & Related papers  (2026-05-24T12:04:29Z)
• Sim-to-Real Transfer for Muscle-Actuated Robots via Generalized Actuator Networks [53.12548325396181]
  Tendon drives paired with soft muscle actuation enable faster and safer robots.<n>We propose a sim-to-real pipeline that learns a neural network model of this complex actuation.<n>We successfully deploy precise goal-reaching and dynamic ball-in-a-cup policies trained entirely in simulation.
  arXiv  Detail & Related papers  (2026-04-10T16:52:54Z)
• Toward AI Autonomous Navigation for Mechanical Thrombectomy using Hierarchical Modular Multi-agent Reinforcement Learning (HM-MARL) [57.65363326406228]
  We propose a Hierarchical Modular Multi-Agent Reinforcement Learning framework for autonomous two-device navigation in vitro.<n>HM-MARL was developed to autonomously navigate a guide catheter and guidewire from the femoral artery to the internal carotid artery (ICA)<n>A modular multi-agent approach was used to decompose the complex navigation task into specialized subtasks, each trained using Soft Actor-Critic RL.<n>In vitro, both HM-MARL models successfully navigated 100% of trials from the femoral artery to the right common carotid artery and 80% to the right ICA but failed on the left-side vessel challenge
  arXiv  Detail & Related papers  (2026-02-20T23:50:35Z)
• Scalable Spatio-Temporal SE(3) Diffusion for Long-Horizon Protein Dynamics [51.85385061275941]
  Molecular dynamics (MD) simulations remain the gold standard for studying protein dynamics.<n>Recent generative models have shown promise in accelerating simulations, yet they struggle with long-horizon generation.<n>We present STAR-MD, a scalable diffusion model that generates physically plausible protein trajectories over micro-scale timescales.
  arXiv  Detail & Related papers  (2026-02-02T14:13:28Z)
• Towards Autonomous Navigation in Endovascular Interventions [0.7100795934780609]
  This thesis presents an integrated AI-driven framework for autonomous guidewire navigation in complex vascular environments.<n>A high-fidelity, real-time simulation platform, CathSim, is introduced for reinforcement learning based catheter navigation.<n>The Expert Navigation Network is developed, a policy that fuses visual, kinematic, and force feedback for autonomous tool control.
  arXiv  Detail & Related papers  (2025-12-19T21:38:52Z)
• Curly Flow Matching for Learning Non-gradient Field Dynamics [49.480209466896035]
  We introduce Curly Flow Matching (Curly-FM), a novel approach to learning non-gradient field dynamics.<n>Curly-FM is capable of learning non-gradient field dynamics by designing and solving a Schr"odinger bridge problem.<n>Curly-FM can learn trajectories that better match both the reference process and population marginals.
  arXiv  Detail & Related papers  (2025-10-30T16:11:39Z)
• Optimal navigation of magnetic artificial microswimmers in blood capillaries with deep reinforcement learning [1.3225694028747141]
  We conduct numerical simulations of artificial bacterial flagella (ABFs) in retinal capillaries propelled by an external magnetic field.<n>The simulations are based on a validated blood model that predicts the dynamics of individual red blood cells and their hydrodynamic interactions with ABFs.<n>We show that the same policy robustly guides the ABF to a prescribed target in both the reduced-order model and the fine-grained blood simulations.
  arXiv  Detail & Related papers  (2024-03-29T17:01:09Z)
• Distilling Reinforcement Learning Policies for Interpretable Robot Locomotion: Gradient Boosting Machines and Symbolic Regression [53.33734159983431]
  This paper introduces a novel approach to distill neural RL policies into more interpretable forms. We train expert neural network policies using RL and distill them into (i) GBMs, (ii) EBMs, and (iii) symbolic policies.
  arXiv  Detail & Related papers  (2024-03-21T11:54:45Z)
• fMRI from EEG is only Deep Learning away: the use of interpretable DL to unravel EEG-fMRI relationships [68.8204255655161]
  We present an interpretable domain grounded solution to recover the activity of several subcortical regions from multichannel EEG data. We recover individual spatial and time-frequency patterns of scalp EEG predictive of the hemodynamic signal in the subcortical nuclei.
  arXiv  Detail & Related papers  (2022-10-23T15:11:37Z)
• Chemotaxis of sea urchin sperm cells through deep reinforcement learning [0.0]
  In this work, we investigate how a model of sea urchin sperm cell can self-learn chemotactic motion in a chemoattractant concentration field. We employ an artificial neural network to act as a decision-making agent and facilitate the sperm cell to discover efficient maneuver strategies. Our results provide insights to the chemotactic process of sea urchin sperm cells and also prepare guidance for the intelligent maneuver of microrobots.
  arXiv  Detail & Related papers  (2022-08-02T06:04:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.