HarnessVLN: Unifying Training-Free Embodied Navigation through an Agent Harness

• URL: http://arxiv.org/abs/2609.15195v1
• Date: Mon, 14 Sep 2026 08:15:56 GMT
• Title: HarnessVLN: Unifying Training-Free Embodied Navigation through an Agent Harness
• Abstract summary: Embodied navigation requires agents to interpret visual observations, accumulate spatial knowledge, and execute actions to follow instructions or locate objects.<n>We present HarnessVLN, a zero-shot, training-free framework whose Agent Harness coordinates perception, retrieval, grounding, navigation, recovery, and termination through a unified tool interface.
• Score: 30.61739879302674
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Embodied navigation requires agents to interpret visual observations, accumulate spatial knowledge, and execute actions to follow instructions or locate objects. Training-based methods face generalization challenges, while training-free methods exploit multimodal large language models (MLLMs) but often lack mechanisms to reconcile proposed actions with spatial evidence, task progress, and execution failures. We present HarnessVLN, a zero-shot, training-free framework whose Agent Harness coordinates perception, retrieval, grounding, navigation, recovery, and termination through a unified tool interface. The Harness validates planner proposals against spatial evidence, geometric feasibility, and subgoal consistency, incorporating structured tool feedback into subsequent decisions. Hierarchical event memory tracks task progress and execution history, while a persistent Spatiotemporal Graph maintains reusable spatial evidence and failure annotations for verification and recovery. A replaceable Navigation Executor converts validated targets into executable motions, allowing the same Harness protocol to support instruction-following and object-goal navigation. HarnessVLN achieves success rates of 60.8%, 53.9%, 76.0%, and 59.3% on R2R, RxR, HM3D-v2, and HM3D-OVON, respectively, surpassing prior training-free SOTA results. Humanoid deployment further demonstrates its applicability to both tasks in real-world environments. The project page is: https://harnessvln.netlify.app/.

Related papers

• Scaffolding Foundation Models into Physical-World Agents Pushes the Frontier of Long-Horizon Navigation [52.90741218586653]
  Long-horizon physical-world agents must reason over distant goals while grounding decisions in reliable closed-loop behavior.<n>We introduce NavMCP, an agentic scaffolding framework that couples a VLM reasoning agent with an NFM executor for long-horizon exploration.<n>NavMCP achieves state-of-the-art results on Embodied Question Answering and a Unitree Go2.
  arXiv  Detail & Related papers  (2026-08-31T07:50:16Z)
• STEGNav: Spatio-Temporal Event Graph Reasoning for Multimodal Lifelong Object Navigation [17.685326212060716]
  We propose a training-free framework that extends conventional scene graphs into event graphs along complementary spatial and temporal axes.<n>STEGNav achieves 66.3% SR and 39.7 SPL on GOAT-Bench, as well as SR scores of 64.0% and 69.4% on HM3D1 and HM3Dv2, respectively.
  arXiv  Detail & Related papers  (2026-08-28T12:42:16Z)
• Embodied-Navigator: Point, Think, Memorize, and Align for Efficient Navigation [68.02899606570223]
  We propose TAMP-Nav, a unified framework for efficient embodied navigation.<n>First, we introduce a Pixel-to-3D Action Formulation (Point) that reformulates navigation into 2D visual coordinates.<n>Second, we propose an integrated Selective Reasoning and Anchor-Trajectory Memory mechanism (Think and Memorize)
  arXiv  Detail & Related papers  (2026-08-18T08:37:08Z)
• PRIMAL3: Pathfinding via Reinforcement and Imitation Multi-Agent Learning - Leveraging LaCAM3 [19.66694371656982]
  PRIMAL3 is an ultra-large-scale framework for multi-agent pathfinding (MAPF)<n>It integrates reinforcement learning, topology-aware communication, LaCAM3-guided training, and PIBT-based action refinement.<n> Experiments demonstrate that PRIMAL3 substantially outperforms state-of-the-art learning-based baselines.
  arXiv  Detail & Related papers  (2026-08-05T14:33:59Z)
• ConsistNav: Closing the Action Consistency Gap in Zero-Shot Object Navigation with Semantic Executive Control [20.155796585668607]
  ConsistNav is a training-free zero-shot ObjectNav framework built around a semantic executive composed of three coordinated modules.<n>We conduct experiments on HM3D and MP3D, where ConsistNav achieves results and improves SR by 11.4% and SPL by 7.9% over the controlled baseline on MP3D.
  arXiv  Detail & Related papers  (2026-05-11T01:51:45Z)
• Stop Wandering: Efficient Vision-Language Navigation via Metacognitive Reasoning [12.880792990007244]
  Training-free Vision-Language Navigation (VLN) agents can follow instructions and explore 3D environments.<n>Existing approaches rely on greedy frontier selection and passive spatial memory.<n>We propose MetaNav, a metacognitive navigation agent integrating spatial memory, history-aware planning, and reflective correction.
  arXiv  Detail & Related papers  (2026-04-02T17:58:08Z)
• ReMemNav: A Rethinking and Memory-Augmented Framework for Zero-Shot Object Navigation [35.416693138335354]
  Zero-shot object navigation requires agents to locate unseen target objects in unfamiliar environments.<n>Recent advancements in vision-language models provide promising commonsense reasoning capabilities for this task.<n>We propose a novel hierarchical navigation framework named ReMemNav, which seamlessly integrates panoramic semantic priors and episodic memory.
  arXiv  Detail & Related papers  (2026-03-25T09:07:32Z)
• Detect Anything via Next Point Prediction [51.55967987350882]
  Rex- Omni is a 3B-scale MLLM that achieves state-of-the-art object perception performance.<n>On benchmarks like COCO and LVIS, Rex- Omni attains performance comparable to or exceeding regression-based models.
  arXiv  Detail & Related papers  (2025-10-14T17:59:54Z)
• Dexplore: Scalable Neural Control for Dexterous Manipulation from Reference-Scoped Exploration [58.4036440289082]
  Hand-object motion-capture (MoCap) offer large-scale, contact-rich demonstrations and hold promise for dexterous robotic scopes.<n>We introduce Dexplore, a unified single-loop optimization that performs repositories and tracking to learn robot control policies directly from MoCap at scale.
  arXiv  Detail & Related papers  (2025-09-11T17:59:07Z)
• Move to Understand a 3D Scene: Bridging Visual Grounding and Exploration for Efficient and Versatile Embodied Navigation [54.04601077224252]
  Embodied scene understanding requires not only comprehending visual-spatial information but also determining where to explore next in the 3D physical world.<n>underlinetextbf3D vision-language learning enables embodied agents to effectively explore and understand their environment.<n>model's versatility enables navigation using diverse input modalities, including categories, language descriptions, and reference images.
  arXiv  Detail & Related papers  (2025-07-05T14:15:52Z)
• Improving Target-driven Visual Navigation with Attention on 3D Spatial Relationships [52.72020203771489]
  We investigate target-driven visual navigation using deep reinforcement learning (DRL) in 3D indoor scenes. Our proposed method combines visual features and 3D spatial representations to learn navigation policy. Our experiments, performed in the AI2-THOR, show that our model outperforms the baselines in both SR and SPL metrics.
  arXiv  Detail & Related papers  (2020-04-29T08:46:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.