Control of Microrobots with Reinforcement Learning under On-Device Compute Constraints

• URL: http://arxiv.org/abs/2512.24740v1
• Date: Wed, 31 Dec 2025 09:18:36 GMT
• Title: Control of Microrobots with Reinforcement Learning under On-Device Compute Constraints
• Abstract summary: An important function of autonomous microrobots is the ability to perform robust movement over terrain.<n>This paper explores an edge ML approach to microrobot locomotion, allowing for on-device, lower latency control under compute, memory, and power constraints.
• Score: 5.456154853270863
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: An important function of autonomous microrobots is the ability to perform robust movement over terrain. This paper explores an edge ML approach to microrobot locomotion, allowing for on-device, lower latency control under compute, memory, and power constraints. This paper explores the locomotion of a sub-centimeter quadrupedal microrobot via reinforcement learning (RL) and deploys the resulting controller on an ultra-small system-on-chip (SoC), SC$μ$M-3C, featuring an ARM Cortex-M0 microcontroller running at 5 MHz. We train a compact FP32 multilayer perceptron (MLP) policy with two hidden layers ($[128, 64]$) in a massively parallel GPU simulation and enhance robustness by utilizing domain randomization over simulation parameters. We then study integer (Int8) quantization (per-tensor and per-feature) to allow for higher inference update rates on our resource-limited hardware, and we connect hardware power budgets to achievable update frequency via a cycles-per-update model for inference on our Cortex-M0. We propose a resource-aware gait scheduling viewpoint: given a device power budget, we can select the gait mode (trot/intermediate/gallop) that maximizes expected RL reward at a corresponding feasible update frequency. Finally, we deploy our MLP policy on a real-world large-scale robot on uneven terrain, qualitatively noting that domain-randomized training can improve out-of-distribution stability. We do not claim real-world large-robot empirical zero-shot transfer in this work.

Related papers

• Nano-U: Efficient Terrain Segmentation for Tiny Robot Navigation [3.5866729725943576]
  We develop a complete framework for robust binary terrain segmentation on a low-cost microcontroller.<n>At the core of our approach we design Nano-U, a highly compact binary segmentation network with a few thousand parameters.<n>We train Nano-U via Quantization-Aware Distillation (QAD), combining knowledge distillation and quantization-aware training.<n>We deploy the quantized Nano-U on a commodity microcontroller by extending MicroFlow, a compiler-based inference engine for TinyML implemented in Rust.
  arXiv  Detail & Related papers  (2026-05-11T08:54:49Z)
• CRoSS: A Continual Robotic Simulation Suite for Scalable Reinforcement Learning with High Task Diversity and Realistic Physics Simulation [46.950823663585425]
  Continual reinforcement learning requires agents to learn from a sequence of tasks without forgetting previously acquired policies.<n>We introduce a novel benchmark suite for CRL based on realistically simulated robots in the Gazebo simulator.
  arXiv  Detail & Related papers  (2026-02-04T18:54:26Z)
• Bridging the Sim-to-Real Gap with multipanda ros2: A Real-Time ROS2 Framework for Multimanual Systems [22.26675117934127]
  We present $multipanda_ros2$, a novel open-source ROS2 architecture for multi-robot control of Franka Robotics robots.<n>Our core contributions address key challenges in real-time torque control, including interaction control and robot-environment modeling.
  arXiv  Detail & Related papers  (2026-02-02T16:11:12Z)
• NanoCockpit: Performance-optimized Application Framework for AI-based Autonomous Nanorobotics [50.594459728605734]
  Small form factor, i.e., a few 10s grams, severely limits onboard computational resources to sub-SI100milliwatt microcontroller units (MCUs)<n>Our framework achieves ideal end-to-end latency, i.e. zero overhead due to serialized tasks, delivering quantifiable improvements in closed-loop control performance.
  arXiv  Detail & Related papers  (2026-01-12T12:29:38Z)
• Learning Sim-to-Real Humanoid Locomotion in 15 Minutes [51.500643119683225]
  We introduce a simple and practical recipe based on off-policy RL algorithms, i.e., FastSAC and FastTD3.<n>Our simple recipe stabilizes off-policy RL algorithms at massive scale with thousands of parallel environments.<n>We demonstrate rapid end-to-end learning of humanoid locomotion controllers on Unitree G1 and Booster T1 robots.
  arXiv  Detail & Related papers  (2025-12-01T18:55:17Z)
• Accelerating TinyML Inference on Microcontrollers through Approximate Kernels [3.566060656925169]
  In this work, we combine approximate computing and software kernel design to accelerate the inference of approximate CNN models on microcontrollers. Our evaluation on an STM32-Nucleo board and 2 popular CNNs trained on the CIFAR-10 dataset shows that, compared to state-of-the-art exact inference, our solutions can feature on average 21% latency reduction.
  arXiv  Detail & Related papers  (2024-09-25T11:10:33Z)
• Training on the Fly: On-device Self-supervised Learning aboard Nano-drones within 20 mW [52.280742520586756]
  Miniaturized cyber-physical systems (CPSes) powered by tiny machine learning (TinyML), such as nano-drones, are becoming an increasingly attractive technology. Simple electronics make these CPSes inexpensive, but strongly limit the computational, memory, and sensing resources available on board. We present a novel on-device fine-tuning approach that relies only on the limited ultra-low power resources available aboard nano-drones.
  arXiv  Detail & Related papers  (2024-08-06T13:11:36Z)
• The Power of Resets in Online Reinforcement Learning [73.64852266145387]
  We explore the power of simulators through online reinforcement learning with local simulator access (or, local planning) We show that MDPs with low coverability can be learned in a sample-efficient fashion with only $Qstar$-realizability. We show that the notorious Exogenous Block MDP problem is tractable under local simulator access.
  arXiv  Detail & Related papers  (2024-04-23T18:09:53Z)
• Reaching the Limit in Autonomous Racing: Optimal Control versus Reinforcement Learning [66.10854214036605]
  A central question in robotics is how to design a control system for an agile mobile robot. We show that a neural network controller trained with reinforcement learning (RL) outperformed optimal control (OC) methods in this setting. Our findings allowed us to push an agile drone to its maximum performance, achieving a peak acceleration greater than 12 times the gravitational acceleration and a peak velocity of 108 kilometers per hour.
  arXiv  Detail & Related papers  (2023-10-17T02:40:27Z)
• Combining model-predictive control and predictive reinforcement learning for stable quadrupedal robot locomotion [0.0]
  We study how this can be achieved by a combination of model-predictive and predictive reinforcement learning controllers. In this work, we combine both control methods to address the quadrupedal robot stable gate generation problem.
  arXiv  Detail & Related papers  (2023-07-15T09:22:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.