MoMa-Kitchen: A 100K+ Benchmark for Affordance-Grounded Last-Mile Navigation in Mobile Manipulation

• URL: http://arxiv.org/abs/2503.11081v1
• Date: Fri, 14 Mar 2025 04:47:38 GMT
• Title: MoMa-Kitchen: A 100K+ Benchmark for Affordance-Grounded Last-Mile Navigation in Mobile Manipulation
• Abstract summary: MoMa-Kitchen is a benchmark dataset for training models to learn optimal final navigation positions.<n>Visual data is collected from RGB-D inputs captured by a first-person view camera mounted on the robotic arm.<n>Our approach enables models to learn affordance-based final positioning that accommodates different arm types and platform heights.
• Score: 46.30498198691935
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In mobile manipulation, navigation and manipulation are often treated as separate problems, resulting in a significant gap between merely approaching an object and engaging with it effectively. Many navigation approaches primarily define success by proximity to the target, often overlooking the necessity for optimal positioning that facilitates subsequent manipulation. To address this, we introduce MoMa-Kitchen, a benchmark dataset comprising over 100k samples that provide training data for models to learn optimal final navigation positions for seamless transition to manipulation. Our dataset includes affordance-grounded floor labels collected from diverse kitchen environments, in which robotic mobile manipulators of different models attempt to grasp target objects amidst clutter. Using a fully automated pipeline, we simulate diverse real-world scenarios and generate affordance labels for optimal manipulation positions. Visual data are collected from RGB-D inputs captured by a first-person view camera mounted on the robotic arm, ensuring consistency in viewpoint during data collection. We also develop a lightweight baseline model, NavAff, for navigation affordance grounding that demonstrates promising performance on the MoMa-Kitchen benchmark. Our approach enables models to learn affordance-based final positioning that accommodates different arm types and platform heights, thereby paving the way for more robust and generalizable integration of navigation and manipulation in embodied AI. Project page: \href{https://momakitchen.github.io/}{https://momakitchen.github.io/}.

Related papers

• LightNav-0: Eliciting VLM Spatial Intelligence for Generalist Embodied Navigation [52.91080323083409]
  Embodied navigation requires agents to translate heterogeneous goals and visual observations into actions across tasks, environments, and robot embodiments.<n>Here we present LightNav-0, a compact generalist embodied navigation model that elicits the spatial intelligence of a pretrained VLM and aligns it with navigation, without task-specific prediction heads.
  arXiv  Detail & Related papers  (2026-08-31T15:08:53Z)
• UniLM-Nav: A Unified Framework for Zero-Shot Last-Mile Navigation [42.39025692205837]
  We propose UniLM-Nav, a framework for zero-shot open-vocabulary last-mile navigation.<n>UniLM-Nav decomposes last-mile navigation into view selection, task-conditioned affordance grounding, and base-pose reasoning.<n>We evaluate UniLM-Nav on the OVMM benchmark, where it outperforms the previous state-of-the-art method, MoTo, by 3.13 percentage points.
  arXiv  Detail & Related papers  (2026-07-07T17:42:08Z)
• Revisiting Articulated Parts Perception in Robot Manipulation [53.043391395657835]
  Geometric Primary Structure (GPS) is an abstraction of the part geometry structure to balance scalability and quality.<n>GPS is integrated with a portable Virtual Reality (VR) device and requires only one minute to annotate one object sequence.<n>With this efficient VR-GPS system, we collect 41K frames for 234 objects across six part classes, and train a generalizable GPS model with a single RGB-D object image as input.
  arXiv  Detail & Related papers  (2026-06-06T11:07:29Z)
• MobileManiBench: Simplifying Model Verification for Mobile Manipulation [70.30578259859512]
  MobileManiBench is a large-scale benchmark for mobile-based robotic manipulation.<n>MobileManiBench features 2 mobile platforms (parallel-gripper and dexterous-hand robots), 2 synchronized cameras (head and right wrist), 630 objects in 20 categories, 5 skills (open, close, pull, push, pick) with over 100 tasks performed in 100 realistic scenes, yielding 300K trajectories.
  arXiv  Detail & Related papers  (2026-02-05T02:49:52Z)
• ReasonNavi: Human-Inspired Global Map Reasoning for Zero-Shot Embodied Navigation [53.95797153529148]
  Embodied agents often struggle with efficient navigation because they rely primarily on partial egocentric observations.<n>We introduce ReasonNavi, a human-inspired framework that operationalizes this reason-then-act paradigm by coupling Multimodal Large Language Models (MLLMs) with deterministic planners.
  arXiv  Detail & Related papers  (2026-01-26T19:09:20Z)
• FeudalNav: A Simple Framework for Visual Navigation [7.136542835931238]
  We develop a hierarchical framework that decomposes the navigation decision-making process into multiple levels.<n>Our method learns to select subgoals through a simple, transferable waypoint selection network.<n>We show competitive results with a suite of SOTA methods in Habitat AI environments without using any odometry in training or inference.
  arXiv  Detail & Related papers  (2026-01-15T22:10:29Z)
• Learning Category-level Last-meter Navigation from RGB Demonstrations of a Single-instance [5.161531917413708]
  Most RGB-based navigation systems only guarantee coarse, meter-level accuracy.<n>This gap prevents manipulation policies from operating within the distribution of their training demonstrations.<n>We introduce an object-centric imitation learning framework for last-meter navigation.
  arXiv  Detail & Related papers  (2025-12-11T23:35:05Z)
• Optimizing Grasping in Legged Robots: A Deep Learning Approach to Loco-Manipulation [0.6533458718563319]
  This paper presents a framework designed to enhance the grasping capabilities of quadrupeds equipped with arms.<n>We developed a pipeline within the Genesis simulation environment to generate a synthetic dataset of grasp attempts on common objects.<n>This dataset was used to train a custom CNN with a U-Net-like architecture that processes multi-modal input from an onboard RGB and depth cameras.<n>We validated the complete framework on a four-legged robot.
  arXiv  Detail & Related papers  (2025-08-24T17:47:56Z)
• CARE: Enhancing Safety of Visual Navigation through Collision Avoidance via Repulsive Estimation [6.216878556851609]
  We propose CARE (Collision Avoidance via Repulsive Estimation) to improve the robustness of learning-based visual navigation methods.<n> CARE can be integrated seamlessly into any RGB-based navigation model that generates local robot trajectories.<n>We evaluate CARE by integrating it with state-of-the-art visual navigation models across diverse robot platforms.
  arXiv  Detail & Related papers  (2025-06-04T11:02:15Z)
• SEMNAV: A Semantic Segmentation-Driven Approach to Visual Semantic Navigation [1.2582887633807602]
  Visual Semantic Navigation (VSN) is a fundamental problem in robotics, where an agent must navigate toward a target object in an unknown environment.<n>Most state-of-the-art VSN models are trained in simulation environments, where rendered scenes of the real world are used, at best.<n>We propose SEMNAV, a novel approach that leverages semantic segmentation as the main visual input representation of the environment.
  arXiv  Detail & Related papers  (2025-06-02T08:19:41Z)
• Scalable Video-to-Dataset Generation for Cross-Platform Mobile Agents [57.59830804627066]
  We introduce MONDAY, a large-scale dataset of 313K annotated frames from 20K instructional videos capturing real-world mobile OS navigation.<n>Models that include MONDAY in their pre-training phases demonstrate robust cross-platform generalization capabilities.<n>We present an automated framework that leverages publicly available video content to create comprehensive task datasets.
  arXiv  Detail & Related papers  (2025-05-19T02:39:03Z)
• A Training-Free Framework for Precise Mobile Manipulation of Small Everyday Objects [16.018172627950857]
  We develop a closed-loop training-free framework that enables a mobile manipulator to tackle precise tasks involving the manipulation of small objects. SVM employs an RGB-D wrist camera and uses visual servoing for control. We demonstrate that open-vocabulary object detectors can serve as a drop-in module to identify semantic targets.
  arXiv  Detail & Related papers  (2025-02-19T18:59:17Z)
• PLANRL: A Motion Planning and Imitation Learning Framework to Bootstrap Reinforcement Learning [13.564676246832544]
  We introduce PLANRL, a framework that chooses when the robot should use classical motion planning and when it should learn a policy. PLANRL switches between two modes of operation: reaching a waypoint using classical techniques when away from the objects and fine-grained manipulation control when about to interact with objects. We evaluate our approach across multiple challenging simulation environments and real-world tasks, demonstrating superior performance in terms of adaptability, efficiency, and generalization compared to existing methods.
  arXiv  Detail & Related papers  (2024-08-07T19:30:08Z)
• Interactive Semantic Map Representation for Skill-based Visual Object Navigation [43.71312386938849]
  This paper introduces a new representation of a scene semantic map formed during the embodied agent interaction with the indoor environment. We have implemented this representation into a full-fledged navigation approach called SkillTron. The proposed approach makes it possible to form both intermediate goals for robot exploration and the final goal for object navigation.
  arXiv  Detail & Related papers  (2023-11-07T16:30:12Z)
• NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration [57.15811390835294]
  This paper describes how we can train a single unified diffusion policy to handle both goal-directed navigation and goal-agnostic exploration. We show that this unified policy results in better overall performance when navigating to visually indicated goals in novel environments. Our experiments, conducted on a real-world mobile robot platform, show effective navigation in unseen environments in comparison with five alternative methods.
  arXiv  Detail & Related papers  (2023-10-11T21:07:14Z)
• Object Goal Navigation with Recursive Implicit Maps [92.6347010295396]
  We propose an implicit spatial map for object goal navigation. Our method significantly outperforms the state of the art on the challenging MP3D dataset. We deploy our model on a real robot and achieve encouraging object goal navigation results in real scenes.
  arXiv  Detail & Related papers  (2023-08-10T14:21:33Z)
• CabiNet: Scaling Neural Collision Detection for Object Rearrangement with Procedural Scene Generation [54.68738348071891]
  We first generate over 650K cluttered scenes - orders of magnitude more than prior work - in diverse everyday environments. We render synthetic partial point clouds from this data and use it to train our CabiNet model architecture. CabiNet is a collision model that accepts object and scene point clouds, captured from a single-view depth observation.
  arXiv  Detail & Related papers  (2023-04-18T21:09:55Z)
• Can an Embodied Agent Find Your "Cat-shaped Mug"? LLM-Guided Exploration for Zero-Shot Object Navigation [58.3480730643517]
  We present LGX, a novel algorithm for Language-Driven Zero-Shot Object Goal Navigation (L-ZSON) Our approach makes use of Large Language Models (LLMs) for this task. We achieve state-of-the-art zero-shot object navigation results on RoboTHOR with a success rate (SR) improvement of over 27% over the current baseline.
  arXiv  Detail & Related papers  (2023-03-06T20:19:19Z)
• PEANUT: Predicting and Navigating to Unseen Targets [18.87376347895365]
  Efficient ObjectGoal navigation (ObjectNav) in novel environments requires an understanding of the spatial and semantic regularities in environment layouts. We present a method for learning these regularities by predicting the locations of unobserved objects from incomplete semantic maps. Our prediction model is lightweight and can be trained in a supervised manner using a relatively small amount of passively collected data.
  arXiv  Detail & Related papers  (2022-12-05T18:58:58Z)
• Image-based Navigation in Real-World Environments via Multiple Mid-level Representations: Fusion Models, Benchmark and Efficient Evaluation [13.207579081178716]
  In recent learning-based navigation approaches, the scene understanding and navigation abilities of the agent are achieved simultaneously. Unfortunately, even if simulators represent an efficient tool to train navigation policies, the resulting models often fail when transferred into the real world. One possible solution is to provide the navigation model with mid-level visual representations containing important domain-invariant properties of the scene.
  arXiv  Detail & Related papers  (2022-02-02T15:00:44Z)
• Vision-Language Navigation with Random Environmental Mixup [112.94609558723518]
  Vision-language Navigation (VLN) tasks require an agent to navigate step-by-step while perceiving the visual observations and comprehending a natural language instruction. Previous works have proposed various data augmentation methods to reduce data bias. We propose the Random Environmental Mixup (REM) method, which generates cross-connected house scenes as augmented data via mixuping environment.
  arXiv  Detail & Related papers  (2021-06-15T04:34:26Z)
• Simultaneous Detection and Tracking with Motion Modelling for Multiple Object Tracking [94.24393546459424]
  We introduce Deep Motion Modeling Network (DMM-Net) that can estimate multiple objects' motion parameters to perform joint detection and association. DMM-Net achieves PR-MOTA score of 12.80 @ 120+ fps for the popular UA-DETRAC challenge, which is better performance and orders of magnitude faster. We also contribute a synthetic large-scale public dataset Omni-MOT for vehicle tracking that provides precise ground-truth annotations.
  arXiv  Detail & Related papers  (2020-08-20T08:05:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.