Scaffolding Foundation Models into Physical-World Agents Pushes the Frontier of Long-Horizon Navigation

• URL: http://arxiv.org/abs/2608.30396v1
• Date: Mon, 31 Aug 2026 07:50:16 GMT
• Title: Scaffolding Foundation Models into Physical-World Agents Pushes the Frontier of Long-Horizon Navigation
• Abstract summary: Long-horizon physical-world agents must reason over distant goals while grounding decisions in reliable closed-loop behavior.<n>We introduce NavMCP, an agentic scaffolding framework that couples a VLM reasoning agent with an NFM executor for long-horizon exploration.<n>NavMCP achieves state-of-the-art results on Embodied Question Answering and a Unitree Go2.
• Score: 52.90741218586653
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long-horizon physical-world agents must reason over distant goals while grounding decisions in reliable closed-loop behavior. Today's foundation models split these capabilities: vision-language models (VLMs) infer missing information and adapt high-level plans but remain brittle and inefficient at repeated navigation grounding, while navigation foundation models (NFMs) robustly execute semantic goals but operate as bounded episodes without persistent task-level reasoning. We introduce NavMCP, an agentic scaffolding framework that couples a VLM reasoning agent with an NFM executor for long-horizon exploration. The VLM decides what evidence to seek, where to search, and when to stop, while the NFM grounds each semantic sub-goal into closed-loop navigation. Three channels structure their collaboration: intent translates evidence needs into navigation calls, observation converts rollouts into source-grounded trajectory evidence, and memory accumulates findings, negative evidence, and unresolved goals across calls. This design turns isolated navigation rollouts into persistent embodied interaction without retraining either model. On Embodied Question Answering, NavMCP achieves state-of-the-art results on HM-EQA, MT-HM3D, and EXPRESS-Bench. Under matched agent and executor backbones, it outperforms an episodic interface by 14.9 percentage points on HM-EQA. On a Unitree Go2, NavMCP reaches 78.3% success, with its margin over the strongest baseline growing from 10 to 45 points as the task horizon increases. These results demonstrate the potential of scaffolding complementary foundation models into long-horizon physical-world agents.

Related papers

• CGFM-Nav: Cognitive Graph-Field Memory for Semantic-Guided Lifelong Multimodal Embodied Navigation [15.24487734405541]
  Vision-and-Language Navigation (VLN) requires agents to reason over accumulated observations while continuously exploring unseen regions.<n>We propose Cognitive Graph-Field Memory (CGFM), a persistent multimodal scene representation.<n>Building upon CGFM, we introduce CGFM-Nav, a foundation-model-based framework for lifelong multimodal navigation.
  arXiv  Detail & Related papers  (2026-08-29T07:54:38Z)
• SSTG-Nav: Metric-Grounded Spatial-Semantic Topological Graphs for Reusable Object Navigation [2.272547247667044]
  SSTG-Nav is a reusable metric-semantic memory that turns a one-time survey into actionable object goals.<n>On 1,000 HM3D-v2 episodes across 36 scenes, our goal-independent topology achieves a 99.4% geometric success ceiling.
  arXiv  Detail & Related papers  (2026-08-01T08:40:04Z)
• Rethinking Embodied Navigation via Relational Inductive Bias [53.72276435022479]
  We propose DB-Nav, a framework that reshapes the search space via dual relational biases.<n>It factorizes target-centric relations into an Activation Bias and an Inhibition Bias.<n>It significantly outperforms existing methods in success rate (SR) and Success weighted by Path Length (SPL)
  arXiv  Detail & Related papers  (2026-06-09T02:57:34Z)
• AnE: Pushing the Reasoning Frontier of Multimodal LLMs via Anchor Evolution [61.593935260052795]
  Post-training viaSupervised Fine-Tuning (SFT) and Reinforcement Learning (RL) is crucial for enhancing reasoning in Multimodal Large Language Models (MLLMs)<n>Existing paradigms often reach a performance bottleneck due to the limitations of static data.<n>We propose Anchor Evolution (AnE), a new paradigm that integrates truth-anchored data curation and model evolution.
  arXiv  Detail & Related papers  (2026-05-25T08:26:34Z)
• HiRO-Nav: Hybrid ReasOning Enables Efficient Embodied Navigation [58.29060712443587]
  HiRO-Nav is the first kind of agent capable of adaptively determining whether to perform thinking at every step based on its own action entropy.<n>We propose a tailored training pipeline comprising hybrid supervised fine-tuning as a cold start, followed by online reinforcement learning.<n>Experiments on the textscCHORES-$mathbbS$ ObjectNav benchmark showcases that HiRO-Nav achieves a better trade-off between success rates and token efficiency than both dense-thinking and no-thinking baselines.
  arXiv  Detail & Related papers  (2026-04-09T13:22:24Z)
• SR-Nav: Spatial Relationships Matter for Zero-shot Object Goal Navigation [23.52929533412924]
  Zero-shot object-goal navigation aims to find target objects in unseen environments using only egocentric observation.<n>Recent methods leverage foundation models' comprehension and reasoning capabilities to enhance navigation performance.<n>We propose Spatial Relation-aware Navigation (SR-Nav), a framework that models both observed and experience-based spatial relationships to enhance both perception and planning.
  arXiv  Detail & Related papers  (2026-03-19T03:09:32Z)
• ReasonNavi: Human-Inspired Global Map Reasoning for Zero-Shot Embodied Navigation [53.95797153529148]
  Embodied agents often struggle with efficient navigation because they rely primarily on partial egocentric observations.<n>We introduce ReasonNavi, a human-inspired framework that operationalizes this reason-then-act paradigm by coupling Multimodal Large Language Models (MLLMs) with deterministic planners.
  arXiv  Detail & Related papers  (2026-01-26T19:09:20Z)
• FOM-Nav: Frontier-Object Maps for Object Goal Navigation [65.76906445210112]
  FOM-Nav is a framework that enhances exploration efficiency through Frontier-Object Maps and vision-language models.<n>To train FOM-Nav, we automatically construct large-scale navigation datasets from real-world scanned environments.<n> FOM-Nav achieves state-of-the-art performance on the MP3D and HM3D benchmarks, particularly in navigation efficiency metric SPL.
  arXiv  Detail & Related papers  (2025-11-30T18:16:09Z)
• Object Navigation with Structure-Semantic Reasoning-Based Multi-level Map and Multimodal Decision-Making LLM [18.406869393228813]
  We propose an active object navigation framework with Environmental Attributes Map (EAM) and MLLM Hierarchical Reasoning module (MHR)<n>EAM is constructed by reasoning observed environments with SBERT and predicting unobserved ones with Diffusion.<n>MHR is inspired by EAM to perform frontier exploration decision-making, avoiding the circuitous trajectories in long-range scenarios to improve path efficiency.
  arXiv  Detail & Related papers  (2025-06-06T09:08:40Z)
• DORAEMON: Decentralized Ontology-aware Reliable Agent with Enhanced Memory Oriented Navigation [55.888688171010365]
  DORAEMON is a cognitive-inspired framework consisting of Ventral and Dorsal Streams that mimics human navigation capabilities.<n>We evaluate DORAEMON on the HM3D, MP3D and GOAT datasets, where it achieves state-of-the-art performance on both success rate (SR) and success weighted by path length (SPL) metrics.
  arXiv  Detail & Related papers  (2025-05-28T04:46:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.