LocoMamba: Vision-Driven Locomotion via End-to-End Deep Reinforcement Learning with Mamba

• URL: http://arxiv.org/abs/2508.11849v2
• Date: Thu, 28 Aug 2025 21:09:41 GMT
• Title: LocoMamba: Vision-Driven Locomotion via End-to-End Deep Reinforcement Learning with Mamba
• Abstract summary: We introduce a vision-driven cross-modal DRL framework built on selective state-space models, specifically leveraging Mamba.<n>LocoMamba achieves near-linear-time sequence modeling, effectively captures long-range dependencies, and enables efficient training with longer sequences.<n>We evaluate our method in challenging simulated environments with static and moving obstacles as well as uneven terrain.
• Score: 2.26464274357758
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: We introduce LocoMamba, a vision-driven cross-modal DRL framework built on selective state-space models, specifically leveraging Mamba, that achieves near-linear-time sequence modeling, effectively captures long-range dependencies, and enables efficient training with longer sequences. First, we embed proprioceptive states with a multilayer perceptron and patchify depth images with a lightweight convolutional neural network, producing compact tokens that improve state representation. Second, stacked Mamba layers fuse these tokens via near-linear-time selective scanning, reducing latency and memory footprint, remaining robust to token length and image resolution, and providing an inductive bias that mitigates overfitting. Third, we train the policy end-to-end with Proximal Policy Optimization under terrain and appearance randomization and an obstacle-density curriculum, using a compact state-centric reward that balances progress, smoothness, and safety. We evaluate our method in challenging simulated environments with static and moving obstacles as well as uneven terrain. Compared with state-of-the-art baselines, our method achieves higher returns and success rates with fewer collisions, exhibits stronger generalization to unseen terrains and obstacle densities, and improves training efficiency by converging in fewer updates under the same compute budget.

Related papers

• SplAttN: Bridging 2D and 3D with Gaussian Soft Splatting and Attention for Point Cloud Completion [17.124815270085048]
  We propose SplAttN, which replaces hard projection with Differentiable Gaussian Splatting to produce a dense, continuous image-plane representation.<n>Experiments show that SplAttN achieves state-of-the-art performance on PCN and ShapeNet-55/34.
  arXiv  Detail & Related papers  (2026-05-02T14:34:21Z)
• LEAR: Learning Edge-Aware Representations for Event-to-LiDAR Localization [15.308350522323588]
  We propose LEAR, a dual-task learning framework that jointly estimates edge structures and dense event-depth flow fields to bridge the sensing-modality divide.<n>On several popular and challenging datasets, LEAR achieves superior performance over the best prior method.
  arXiv  Detail & Related papers  (2026-03-02T13:18:25Z)
• Event-based Visual Deformation Measurement [76.25283405575108]
  Visual Deformation Measurement aims to recover dense deformation fields by tracking surface motion from camera observations.<n>Traditional image-based methods rely on minimal inter-frame motion to constrain the correspondence search space.<n>We propose an event-frame fusion framework that exploits events for temporally dense motion cues and frames for spatially dense precise estimation.
  arXiv  Detail & Related papers  (2026-02-16T01:04:48Z)
• MambaEye: A Size-Agnostic Visual Encoder with Causal Sequential Processing [14.888533532729864]
  MambaEye is a novel, causal sequential encoder that leverages the low complexity and causal-process based pure Mamba2 backbone.<n>Unlike previous Mamba-based vision encoders that often employ bidirectional processing, our strictly unidirectional approach preserves the inherent causality of State Space Models.<n>MambaEye exhibits robust performance across a wide range of image resolutions, especially at higher resolutions such as $15362$ on the ImageNet-1K classification task.
  arXiv  Detail & Related papers  (2025-11-25T06:18:18Z)
• Parameter Interpolation Adversarial Training for Robust Image Classification [14.913267679379308]
  We propose a novel framework called Interpolation Adversarial Training (PIAT)<n> PIAT tunes the model parameters between each epoch by interpolating the parameters of the previous and current epochs.<n>It makes the decision boundary of model change more moderate and alleviates the overfitting issue.
  arXiv  Detail & Related papers  (2025-11-02T07:37:06Z)
• Scaling Up Occupancy-centric Driving Scene Generation: Dataset and Method [54.461213497603154]
  Occupancy-centric methods have recently achieved state-of-the-art results by offering consistent conditioning across frames and modalities.<n>Nuplan-Occ is the largest occupancy dataset to date, constructed from the widely used Nuplan benchmark.<n>We develop a unified framework that jointly synthesizes high-quality occupancy, multi-view videos, and LiDAR point clouds.
  arXiv  Detail & Related papers  (2025-10-27T03:52:45Z)
• BADiff: Bandwidth Adaptive Diffusion Model [55.10134744772338]
  Traditional diffusion models produce high-fidelity images by performing a fixed number of denoising steps, regardless of downstream transmission limitations.<n>In practical cloud-to-device scenarios, limited bandwidth often necessitates heavy compression, leading to loss of fine textures and wasted computation.<n>We introduce a joint end-to-end training strategy where the diffusion model is conditioned on a target quality level derived from the available bandwidth.
  arXiv  Detail & Related papers  (2025-10-24T11:50:03Z)
• Can SSD-Mamba2 Unlock Reinforcement Learning for End-to-End Motion Control? [2.211616153144189]
  We present a vision-driven cross-modal framework built on SSD-Mamba2, a selective state-space backbone.<n>Proprioceptive states and exteroceptive observations are encoded into compact tokens and fused by stacked SSD-Mamba2 layers.<n>Our approach consistently surpasses strong state-of-the-art baselines in return, safety, and sample efficiency.
  arXiv  Detail & Related papers  (2025-09-09T11:05:44Z)
• MMAR: Towards Lossless Multi-Modal Auto-Regressive Probabilistic Modeling [64.09238330331195]
  We propose a novel Multi-Modal Auto-Regressive (MMAR) probabilistic modeling framework.<n>Unlike discretization line of method, MMAR takes in continuous-valued image tokens to avoid information loss in an efficient way.<n>We also propose a theoretically proven technique that addresses the numerical stability issue and a training strategy that balances the generation and understanding task goals.
  arXiv  Detail & Related papers  (2024-10-14T17:57:18Z)
• Coherent and Multi-modality Image Inpainting via Latent Space Optimization [61.99406669027195]
  PILOT (intextbfPainting vtextbfIa textbfLatent textbfOptextbfTimization) is an optimization approach grounded on a novel textitsemantic centralization and textitbackground preservation loss. Our method searches latent spaces capable of generating inpainted regions that exhibit high fidelity to user-provided prompts while maintaining coherence with the background.
  arXiv  Detail & Related papers  (2024-07-10T19:58:04Z)
• DynaSeg: A Deep Dynamic Fusion Method for Unsupervised Image Segmentation Incorporating Feature Similarity and Spatial Continuity [0.5755004576310334]
  We introduce DynaSeg, an innovative unsupervised image segmentation approach. Unlike traditional methods, DynaSeg employs a dynamic weighting scheme that adapts flexibly to image characteristics. DynaSeg prevents undersegmentation failures where the number of predicted clusters might converge to one.
  arXiv  Detail & Related papers  (2024-05-09T00:30:45Z)
• Taming Contrast Maximization for Learning Sequential, Low-latency, Event-based Optical Flow [18.335337530059867]
  Event cameras have gained significant traction since they open up new avenues for low-latency and low-power solutions to complex computer vision problems. To unlock these solutions, it is necessary to develop algorithms that can leverage the unique nature of event data. In this work, we propose a novel self-supervised learning pipeline for the estimation of event-based optical flow.
  arXiv  Detail & Related papers  (2023-03-09T12:37:33Z)
• Model-Based Reinforcement Learning via Latent-Space Collocation [110.04005442935828]
  We argue that it is easier to solve long-horizon tasks by planning sequences of states rather than just actions. We adapt the idea of collocation, which has shown good results on long-horizon tasks in optimal control literature, to the image-based setting by utilizing learned latent state space models.
  arXiv  Detail & Related papers  (2021-06-24T17:59:18Z)
• An Adaptive Framework for Learning Unsupervised Depth Completion [59.17364202590475]
  We present a method to infer a dense depth map from a color image and associated sparse depth measurements. We show that regularization and co-visibility are related via the fitness of the model to data and can be unified into a single framework.
  arXiv  Detail & Related papers  (2021-06-06T02:27:55Z)
• Self-Supervised Multi-Frame Monocular Scene Flow [61.588808225321735]
  We introduce a multi-frame monocular scene flow network based on self-supervised learning. We observe state-of-the-art accuracy among monocular scene flow methods based on self-supervised learning.
  arXiv  Detail & Related papers  (2021-05-05T17:49:55Z)
• Stylized Adversarial Defense [105.88250594033053]
  adversarial training creates perturbation patterns and includes them in the training set to robustify the model. We propose to exploit additional information from the feature space to craft stronger adversaries. Our adversarial training approach demonstrates strong robustness compared to state-of-the-art defenses.
  arXiv  Detail & Related papers  (2020-07-29T08:38:10Z)
• Perturbing Across the Feature Hierarchy to Improve Standard and Strict Blackbox Attack Transferability [100.91186458516941]
  We consider the blackbox transfer-based targeted adversarial attack threat model in the realm of deep neural network (DNN) image classifiers. We design a flexible attack framework that allows for multi-layer perturbations and demonstrates state-of-the-art targeted transfer performance. We analyze why the proposed methods outperform existing attack strategies and show an extension of the method in the case when limited queries to the blackbox model are allowed.
  arXiv  Detail & Related papers  (2020-04-29T16:00:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.