Micro-Swarm Locomotion Optimization in Dynamic Flow using Multi-Objective Multi-Agent Reinforcement Learning

• URL: http://arxiv.org/abs/2605.25025v1
• Date: Sun, 24 May 2026 12:04:29 GMT
• Title: Micro-Swarm Locomotion Optimization in Dynamic Flow using Multi-Objective Multi-Agent Reinforcement Learning
• Abstract summary: Coordinating micro-robotic swarms in physiologically realistic, time-dependent fluid environments remains an unsolved challenge.<n>We present a hybrid Computational Fluid Dynamics - Multi-Objective Multi-Agent Reinforcement Learning framework.<n>16 magnetically actuated micro-robots navigate a pulsatile arterial, simultaneously optimizing upstream progression, energy conservation, and motion smoothness.
• Score: 2.0223202343873226
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Coordinating micro-robotic swarms in physiologically realistic, time-dependent fluid environments remains an unsolved challenge for biomedical and environmental applications. We present a hybrid Computational Fluid Dynamics - Multi-Objective Multi-Agent Reinforcement Learning framework that directly couples a high-fidelity incompressible Navier-Stokes solver with decentralized proximal policy optimization to learn physically consistent swarm control strategies in oscillatory flow. Sixteen magnetically actuated micro-robots navigate a pulsatile arterial waveform, simultaneously optimizing upstream progression, energy conservation, and motion smoothness, reconciled using PCGrad surgery. Without PCGrad, energy efficiency and smoothness rewards collapse to near zero within 10,000 training steps while progress exhibits persistent large-amplitude oscillations, confirming that gradient conflict resolution is a structural requirement rather than an optional refinement in this domain. The converged policy achieves a progress reward of 6.5-7.0, a sustained energy efficiency of 0.63-0.65, and near-maximum smoothness (0.97-0.99), representing improvements over brute-force baselines on the primary objective while both baselines yield negative energy efficiency throughout. Training reveals three emergent behavioral phases: a collective two-layer hydrodynamic throttling formation that suppresses peak channel velocities during forward flow, a cycle-synchronized ratchet mechanism that exploits flow reversals for upstream repositioning, and an individualized final approach as agents near the success boundary. These results establish that time-dependent fluid-agent interactions can be captured directly within multi-objective reinforcement learning loops, offering a physically grounded paradigm for micro-swarm control in biomedical navigation, environmental monitoring, and industrial microfluidics.

Related papers

• Flow-aware Optimal Navigation in Unsteady Flows through Reinforcement Learning [0.815557531820863]
  We present a reinforcement learning approach to train autonomous agents to reach arbitrary targets in a chaotic double-gyre flow.<n>We analyze the impact of providing agents with explicit global flow parameters.<n>The experiments reveal a trade-off in sensor utility: velocity-aware agents optimize energy efficiency, whereas vorticity sensors provide superior structural mapping and achieve better target proximity.
  arXiv  Detail & Related papers  (2026-07-15T07:57:26Z)
• Reinforcement Learning Enables Autonomous Microrobot Navigation and Intervention in Simulated Blood Capillaries [0.903934726553365]
  We develop a physically grounded simulation of a blood capillary network.<n>We train deep reinforcement learning agents to navigate it via chemotaxis.<n>These results establish RL as a viable framework for developing autonomous microrobotic intervention strategies in complex biological environments.
  arXiv  Detail & Related papers  (2026-06-23T12:21:18Z)
• Invertible Neural Network Adapter for One-Step Flow Matching in Robot Manipulation [48.77928097573051]
  This paper presents an invertible neural network adapter for general robotic manipulation.<n>Built upon a flow-matching formulation, the proposed adapter constrains the action generation trajectory within an invertible latent space.<n>Experiments are conducted across a diverse set of simulation benchmarks and real-world robotic platforms to evaluate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2026-06-17T15:35:27Z)
• Learning Control as Enabling Layer for Embodied Intelligence Research explored with Soft Robotic Swimming in diverse Flow Speeds [0.0]
  We evaluate whether augmenting PID control with a Linear Repetitive Learning Estimation Scheme (PID-LRLES) recovers tracking accuracy and repeatability under dynamic flow.<n>The LRLES generalizes classical integral action from constant to periodic, non-constant references, while using a stable transfer-function realization.<n>This approach provides an enabling layer for future robophysical studies seeking to isolate the effects of morphology, sensing, and environmental flow on aquatic locomotion.
  arXiv  Detail & Related papers  (2026-06-09T15:20:13Z)
• Learn Weightlessness: Imitate Non-Self-Stabilizing Motions on Humanoid Robot [56.377161933570335]
  Humans naturally exploit a "weightless" state during non-self-stabilizing (NSS) motions.<n>Inspired by this biological mechanism, we propose a weightlessness-state auto-labeling strategy for dataset annotation.<n>Our work bridges the gap between precise trajectory tracking and adaptive environmental interaction, offering a biologically-inspired solution for contact-rich humanoid control.
  arXiv  Detail & Related papers  (2026-04-23T07:10:05Z)
• Dynamic Modeling and Robust Gait Optimization of a Compliant Worm Robot [6.048766517808481]
  This paper presents an experimentally grounded modeling and optimization framework for a compliant worm robot capable of traversing corrugated pipes.<n>A multi-objective gait optimization problem is formulated to maximize average speed while minimizing average power.<n>Results show that the proposed framework captures the dominant locomotion and energy-consumption behavior of the robot.
  arXiv  Detail & Related papers  (2026-04-13T20:20:03Z)
• MIOFlow 2.0: A unified framework for inferring cellular stochastic dynamics from single cell and spatial transcriptomics data [47.97758608694957]
  We introduce Interpolating Manifold-Transport Flow (MIOFlow) 2.0.<n>This framework learns biologically informed cellular trajectories by integrating manifold learning, optimal transport, and neural differential equations.<n>It is validated on synthetic datasets, embryoid body differentiation, and spatially resolved axolotl brain regeneration.
  arXiv  Detail & Related papers  (2026-03-23T20:49:45Z)
• Hebbian Attractor Networks for Robot Locomotion [44.84033948079516]
  Hebbian plasticity offers a promising path toward rapid adaptation in changing environments.<n>Hebbian Attractor Networks (HANs) induce emergent attractor dynamics.<n>Results show that slower updates, combined with averaged pre- and postsynaptic activations, promote convergence to stable weight configurations.
  arXiv  Detail & Related papers  (2026-03-23T19:20:14Z)
• Ada3Drift: Adaptive Training-Time Drifting for One-Step 3D Visuomotor Robotic Manipulation [53.750389076941396]
  Diffusion-based visuomotor policies capture multimodal action distributions through iterative denoising, but their high inference latency limits real-time robotic control.<n>We propose Ada3Drift, which learns a training-time drifting field that attracts predicted actions toward expert demonstration modes.<n>Ada3Drift achieves state-of-the-art performance while requiring $10times$ fewer function evaluations than diffusion-based alternatives.
  arXiv  Detail & Related papers  (2026-03-12T14:36:53Z)
• FISC: A Fluid-Inspired Framework for Decentralized and Scalable Swarm Control [0.0]
  A decentralized approach for outer-loop control of large multi-agent systems is proposed and evaluated.<n>A relationship between fundamental fluidic element properties and individual robotic agent states is developed.<n>By ascribing fluid-like properties to subsets of agents, the swarm evolves collectively while maintaining desirable structure and coherence.<n>Results demonstrate the feasibility of treating large robotic swarms as continuum systems that retain the macroscopic structure derived from first principles.
  arXiv  Detail & Related papers  (2026-01-31T03:04:38Z)
• CoLA-Flow Policy: Temporally Coherent Imitation Learning via Continuous Latent Action Flow Matching for Robotic Manipulation [4.726851899243877]
  LG-Flow Policy is a trajectory-level imitation learning framework that performs flow matching in a continuous latent action space.<n>By encoding action sequences into temporally regularized latent trajectories and learning an explicit latent-space flow, the proposed approach decouples global motion structure from low-level control noise.
  arXiv  Detail & Related papers  (2026-01-30T15:36:43Z)
• Multi-Scale Temporal Homeostasis Enables Efficient and Robust Neural Networks [0.0]
  Multi-Scale Temporal Homeostasis (MSTH) is a framework that integrates ultra-fast (5-ms), fast (2-s), medium (5-min) and slow (1-hrs) regulation into artificial networks.<n>MSTH consistently improves accuracy, eliminates catastrophic failures and enhances recovery from perturbations.
  arXiv  Detail & Related papers  (2026-01-30T13:33:29Z)
• Curly Flow Matching for Learning Non-gradient Field Dynamics [49.480209466896035]
  We introduce Curly Flow Matching (Curly-FM), a novel approach to learning non-gradient field dynamics.<n>Curly-FM is capable of learning non-gradient field dynamics by designing and solving a Schr"odinger bridge problem.<n>Curly-FM can learn trajectories that better match both the reference process and population marginals.
  arXiv  Detail & Related papers  (2025-10-30T16:11:39Z)
• Nauplius Optimisation for Autonomous Hydrodynamics [0.0]
  NOAH is a novel nature-inspired swarm optimisation algorithm.<n>It combines current-aware drift, irreversible settlement in persistent sensing nodes, and colony-based communication.<n> validation studies demonstrate an 86% success rate for permanent anchoring scenarios.
  arXiv  Detail & Related papers  (2025-10-17T06:30:19Z)
• Learning to Move in Rhythm: Task-Conditioned Motion Policies with Orbital Stability Guarantees [45.137864140049814]
  We introduce Orbitally Stable Motion Primitives (OSMPs) - a framework that combines a learned diffeomorphic encoder with a supercritical Hopf bifurcation in latent space.<n>We validate the proposed approach through extensive simulation and real-world experiments across a diverse range of robotic platforms.
  arXiv  Detail & Related papers  (2025-07-12T17:10:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.