Learning to Evolve: Multi-modal Interactive Fields for Robust Humanoid Navigation in Dynamic Environments

• URL: http://arxiv.org/abs/2605.21935v1
• Date: Thu, 21 May 2026 03:11:43 GMT
• Title: Learning to Evolve: Multi-modal Interactive Fields for Robust Humanoid Navigation in Dynamic Environments
• Abstract summary: Multi-modal Interactive Field (MIF) is a humanoid-oriented system that integrates confidence-aware semantic 3D Gaussian Splatting, discrepancy-triggered spatial memory updates, and task-driven geometric reconstruction within a closed-loop perception-adaptation pipeline.<n>On a Unitree-G1 humanoid in a real dynamic office, MIF improves relocation success in non-static environments from 12% to 94% compared with static scene-graph memory, while reducing semantic memory footprint by 91.4% through feature distillation for practical online operation.
• Score: 10.149525023566712
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Safe manipulation-oriented navigation for humanoid robots requires scene memory that remains reliable under locomotion-induced perceptual distortion, environmental changes, and interaction-level geometric safety constraints. Existing semantic mapping and scene-graph systems are difficult to deploy directly in this setting because they often assume stable camera trajectories, static environments, or coarse object geometry. We introduce the Multi-modal Interactive Field (MIF), a humanoid-oriented system that integrates confidence-aware semantic 3D Gaussian Splatting, discrepancy-triggered spatial memory updates, and task-driven geometric reconstruction within a closed-loop perception-adaptation pipeline. MIF couples three fields: an uncertainty-aware 3DGS Appearance Field that suppresses gait-induced blur, a Spatial Field that maintains topological memory, and a Geometry Field that supports Interaction Pose Safety (IPS) before manipulation. A discrepancy detection score is introduced to separate locomotion-induced false-positive changes from persistent changes and updates only locally inconsistent regions. On a Unitree-G1 humanoid in a real dynamic office, MIF improves relocation success in non-static environments from 12% to 94% compared with static scene-graph memory, while reducing semantic memory footprint by 91.4% through feature distillation for practical online operation. Project page and code: https://ziya-jiang.github.io/MIF-homepage/

Related papers

• HitMem: Hierarchical Temporal 3D Memory with Multi-Modal Context-Aware Retrieval for Dynamic Environments [5.55387168911911]
  HitMem is a hierarchical temporal 3D memory framework with a multi-modal context-aware retrieval mechanism.<n>We show that HitMem significantly improves object relocation accuracy, reduces exploration costs, and enhances task execution performance in dynamic environments.
  arXiv  Detail & Related papers  (2026-09-01T09:09:21Z)
• Lifelong Localization in Dynamic Indoor Environments Combining Odometry with Sparse Distance Sampling [5.257115841810258]
  We propose a robust lifelong localization framework in dynamic indoor environments.<n>We demonstrate how distance samples can be used to provide a robust prior on the robot's location.<n>We achieve a localization comparable to SLAM with merely a few (sixteen) distance samples, as opposed to the full LiDAR range.
  arXiv  Detail & Related papers  (2026-07-20T11:46:57Z)
• SplatCtrl: Perception-Action Coupling via Gaussian Scene Representations and Reactive Robot Control [14.952269223197751]
  This paper presents SplatCtrl, a unified framework for real-time scene reconstruction and reactive robot motion generation.<n>Building on 3D Gaussian Splatting (3D-GS), we introduce a hybrid voxel-based filtering and dynamic Gaussian relocation strategy.<n>For safe and reactive control, we propose a method for deriving continuous signed distance functions from isotropic Gaussians.
  arXiv  Detail & Related papers  (2026-07-09T21:18:39Z)
• PhyScene3D: Physically Consistent Interactive 3D Tabletop Scene Generation [75.9028457575434]
  Generating 3D tabletop scenes is a fundamental yet underexplored problem for interactive and generalist robotic learning.<n>Here, we introduce PhyScene3D, a framework that reformulates generation as a Human-Mimetic Constructive Process.<n>Experiments demonstrate that PhyScene3D outperforms state-of-the-art approaches in both semantic accuracy and physical validity.
  arXiv  Detail & Related papers  (2026-06-01T03:56:32Z)
• Dynamic Resilient Spatio-Semantic Memory with Hybrid Localization for Mobile Manipulation [13.692484702750313]
  This paper presents DREAM, a real-robot mobile manipulation framework that integrates memory, localization, navigation, and manipulation in previously unseen indoor environments without a pre-built map.<n>For target localization and reacquisition, DREAM combines language-conditioned 3D retrieval, open-vocabulary image detection, and multimodal large language model based semantic verification.<n>Real-robot experiments in four dynamic indoor laboratory scenes show that DREAM improves long-horizon task success rates from 40%-60% with DynaMem to 55%-70%, while maintaining a memory footprint of 0.37-0.63 GB and an online memory-update time of 0.43-0.53
  arXiv  Detail & Related papers  (2026-05-30T06:58:03Z)
• Change-Robust Online Spatial-Semantic Topological Mapping [10.541004576272163]
  Autonomous robots require change-robust-semantic reasoning to decide where to go, how to get there, and where the robot is despite environmental change.<n>Existing approaches typically attach semantics to SLAM-built metric maps, but these pipelines are brittle under appearance shifts and scene dynamics.<n>We propose a Change-Robust Online Spatial-Semantic representation that replaces a globally consistent metric substrate with an online, pose-aware topological graph of RGB-D rearrangements.
  arXiv  Detail & Related papers  (2026-05-04T05:00:12Z)
• Robust Cross-Domain WiFi Fall Detection via Physics-Driven Attention-Enhanced Transformers [64.35403680946182]
  Device-free fall detection utilizing WiFi Channel State Information (CSI) has emerged as a promising solution for elderly health monitoring.<n>Existing deep learning approaches suffer from severe performance degradation when deployed in unseen environments.<n>We propose a robust, domain-general framework featuring a novel Attention-Enhanced CNN-Transformer hybrid architecture.
  arXiv  Detail & Related papers  (2026-04-23T03:28:12Z)
• MeshMimic: Geometry-Aware Humanoid Motion Learning through 3D Scene Reconstruction [54.36564144414704]
  MeshMimic is an innovative framework that bridges 3D scene reconstruction and embodied intelligence to enable humanoid robots to learn coupled "motion-terrain" interactions directly from video.<n>By leveraging state-of-the-art 3D vision models, our framework precisely segments and reconstructs both human trajectories and the underlying 3D geometry of terrains and objects.
  arXiv  Detail & Related papers  (2026-02-17T17:09:45Z)
• SpatialActor: Exploring Disentangled Spatial Representations for Robust Robotic Manipulation [63.48859753472547]
  SpatialActor is a framework for robust robotic manipulation that explicitly decouples semantics and geometry.<n>It achieves state-of-the-art performance with 87.4% on RLBench and improves by 13.9% to 19.4% under varying noisy conditions.
  arXiv  Detail & Related papers  (2025-11-12T18:59:08Z)
• HOSIG: Full-Body Human-Object-Scene Interaction Generation with Hierarchical Scene Perception [57.37135310143126]
  HO SIG is a novel framework for synthesizing full-body interactions through hierarchical scene perception.<n>Our framework supports unlimited motion length through autoregressive generation and requires minimal manual intervention.<n>This work bridges the critical gap between scene-aware navigation and dexterous object manipulation.
  arXiv  Detail & Related papers  (2025-06-02T12:08:08Z)
• OMEGA: Efficient Occlusion-Aware Navigation for Air-Ground Robot in Dynamic Environments via State Space Model [12.096387853748938]
  Air-ground robots (AGRs) are widely used in surveillance and disaster response.<n>Current AGR navigation systems perform well in static environments.<n>However, these systems face challenges in dynamic, severe occlusion scenes.<n>We propose OccMamba with an Efficient AGR-Planner to address these problems.
  arXiv  Detail & Related papers  (2024-08-20T07:50:29Z)
• Memorize What Matters: Emergent Scene Decomposition from Multitraverse [54.487589469432706]
  We introduce 3D Gaussian Mapping, a camera-only offline mapping framework grounded in 3D Gaussian Splatting. 3DGM converts multitraverse RGB videos from the same region into a Gaussian-based environmental map while concurrently performing 2D ephemeral object segmentation. We build the Mapverse benchmark, sourced from the Ithaca365 and nuPlan datasets, to evaluate our method in unsupervised 2D segmentation, 3D reconstruction, and neural rendering.
  arXiv  Detail & Related papers  (2024-05-27T14:11:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.