LLMs for Text-Based Exploration and Navigation Under Partial Observability

• URL: http://arxiv.org/abs/2604.09604v1
• Date: Tue, 10 Mar 2026 10:38:47 GMT
• Title: LLMs for Text-Based Exploration and Navigation Under Partial Observability
• Abstract summary: Exploration and goal-directed navigation in unknown layouts are central to inspection, logistics, and search-and-rescue.<n>We ask whether large language models (LLMs) can function as emphtext-only controllers under partial observability.
• Score: 0.8478633389228728
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Exploration and goal-directed navigation in unknown layouts are central to inspection, logistics, and search-and-rescue. We ask whether large language models (LLMs) can function as \emph{text-only} controllers under partial observability -- without code execution, tools, or program synthesis. We introduce a reproducible benchmark with oracle localisation in fixed ASCII gridworlds: each step reveals only a local $5\times5$ window around the agent and the model must select one of \texttt{UP/RIGHT/DOWN/LEFT}. Nine contemporary LLMs ranging from open/proprietary, dense / Mixture of Experts and instruction- vs. reasoning-tuned are evaluated on two tasks across three layouts of increasing difficulty: \emph{Exploration} (maximising revealed cells) and \emph{Navigation} (reach the goal on the shortest path). The experimental results are evaluated on quantitative metrics including \emph{success rate}, \emph{efficiency} such as normalised coverage and \emph{path length} vs. oracle as well as qualitative analysis. Reasoning-tuned models reliably complete navigation across all layouts, yet remain less efficient than oracle paths. Few-shot demonstrations in the prompt chiefly help these Reasoning-tuned models by reducing invalid moves and shortening paths, while classic dense instruction models remain inconsistent. We observe characteristic action priors (UP/RIGHT) that can induce looping under partial observability. Overall, training regimen and test-time deliberation predict control ability better than raw parameter count. These findings suggest lightweight hybridisation with classical online planners as a practical route to deployable partial map systems.

Related papers

• Active Exploring like a Pigeon: Reinforcing Spatial Reasoning via Agentic Vision-Language Models [17.24700274670692]
  Existing approaches treat spatial language models (VLMs) as passive observers, which is difficult for real-world applications.<n>Inspired by pigeons' building and exploiting cognitive maps for navigation, we propose a novel agentic pipeline for spatial reasoning.<n>Experiments on VisionCube benchmark demonstrate state-of-the-art performance with emph80.5% overall accuracy, outperforming the best current method by emph29.5 accuracy points.
  arXiv  Detail & Related papers  (2026-06-01T16:30:56Z)
• LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling [63.679448814185456]
  Test-time scaling (TTS) has become an effective approach for improving large language model performance.<n>Existing TTS strategies are largely hand-crafted: researchers design patterns and tune them by intuition, leaving much of the computation-allocation space unexplored.<n>We propose an environment-driven framework, AutoTTS, that changes what researchers design: from individual TTSs to environments where TTS strategies can be discovered automatically.
  arXiv  Detail & Related papers  (2026-05-08T17:59:40Z)
• DocSeeker: Structured Visual Reasoning with Evidence Grounding for Long Document Understanding [63.257540233507626]
  We propose a paradigm that requires the model to execute a structured Analysis, Localization and Reasoning workflow.<n>We show it robustly generalizes from short-page training to ultra-long documents and is naturally synergistic with visual Retrieval-Augmented Generation systems.
  arXiv  Detail & Related papers  (2026-04-14T14:39:26Z)
• STAR: Semantic-Tuned and Tail-Adaptive Retriever for Graph-Augmented Generation [69.66045538889777]
  We propose STAR, a semantic-tuned and tail-adaptive retriever for GraphRAG.<n> STAR consistently outperforms baselines, achieving average retrieval performance gains of 1.8%.
  arXiv  Detail & Related papers  (2026-04-11T10:16:51Z)
• ReLope: KL-Regularized LoRA Probes for Multimodal LLM Routing [5.106806600400179]
  We introduce two complementary approaches for improving probe routing in large language model (LLM) systems.<n>First, we propose the emphAttention Probe, which aggregates hidden states from the preceding layer based on attention scores to recover distributed correctness signals.<n>Second, we present the emphKL-Regularized LoRA Probe (ReLope), which inserts a lightweight LoRA adapter and applies a KL regularizer to learn routing-aware representations.
  arXiv  Detail & Related papers  (2026-03-25T20:00:57Z)
• ReasonNavi: Human-Inspired Global Map Reasoning for Zero-Shot Embodied Navigation [53.95797153529148]
  Embodied agents often struggle with efficient navigation because they rely primarily on partial egocentric observations.<n>We introduce ReasonNavi, a human-inspired framework that operationalizes this reason-then-act paradigm by coupling Multimodal Large Language Models (MLLMs) with deterministic planners.
  arXiv  Detail & Related papers  (2026-01-26T19:09:20Z)
• In-Context Operator Learning on the Space of Probability Measures [11.178575236157961]
  We introduce emphin-context operator learning on probability measure spaces for optimal transport.<n>We parameterize the solution operator and develop scaling-law theory in two regimes.<n>Our numerical experiments on synthetic transports and generative-modeling benchmarks validate the framework.
  arXiv  Detail & Related papers  (2026-01-15T01:44:10Z)
• Follow the Signs: Using Textual Cues and LLMs to Guide Efficient Robot Navigation [8.782169490089673]
  We propose a novel semantic navigation framework that infers patterns from partial observations and predicts regions where the goal is most likely located.<n>Our method combines local perceptual inputs with frontier-based exploration and periodic LLM queries.<n>We demonstrate that this approach enables more efficient navigation in sparse, partially observable grid environments by exploiting symbolic patterns.
  arXiv  Detail & Related papers  (2026-01-10T18:47:25Z)
• LLM-First Search: Self-Guided Exploration of the Solution Space [29.780554400938335]
  Large Language Models (LLMs) have demonstrated remarkable improvements in reasoning and planning through increased test-time compute.<n>We propose textbfLLM-First Search (LFS), a novel textitLLM Self-Guided Search method.
  arXiv  Detail & Related papers  (2025-06-05T16:27:49Z)
• Semi-DETR: Semi-Supervised Object Detection with Detection Transformers [105.45018934087076]
  We analyze the DETR-based framework on semi-supervised object detection (SSOD) We present Semi-DETR, the first transformer-based end-to-end semi-supervised object detector. Our method outperforms all state-of-the-art methods by clear margins.
  arXiv  Detail & Related papers  (2023-07-16T16:32:14Z)
• Single-Stage Visual Relationship Learning using Conditional Queries [60.90880759475021]
  TraCQ is a new formulation for scene graph generation that avoids the multi-task learning problem and the entity pair distribution. We employ a DETR-based encoder-decoder conditional queries to significantly reduce the entity label space as well. Experimental results show that TraCQ not only outperforms existing single-stage scene graph generation methods, it also beats many state-of-the-art two-stage methods on the Visual Genome dataset.
  arXiv  Detail & Related papers  (2023-06-09T06:02:01Z)
• Parameter-efficient Tuning of Large-scale Multimodal Foundation Model [68.24510810095802]
  We propose A graceful prompt framework for cross-modal transfer (Aurora) to overcome these challenges. Considering the redundancy in existing architectures, we first utilize the mode approximation to generate 0.1M trainable parameters to implement the multimodal prompt tuning. A thorough evaluation on six cross-modal benchmarks shows that it not only outperforms the state-of-the-art but even outperforms the full fine-tuning approach.
  arXiv  Detail & Related papers  (2023-05-15T06:40:56Z)
• Learning to Predict Navigational Patterns from Partial Observations [63.04492958425066]
  This paper presents the first self-supervised learning (SSL) method for learning to infer navigational patterns in real-world environments from partial observations only. We demonstrate how to infer global navigational patterns by fitting a maximum likelihood graph to the DSLP field. Experiments show that our SSL model outperforms two SOTA supervised lane graph prediction models on the nuScenes dataset.
  arXiv  Detail & Related papers  (2023-04-26T02:08:46Z)
• Guiding Large Language Models via Directional Stimulus Prompting [114.84930073977672]
  We introduce Directional Stimulus Prompting, a novel framework for guiding black-box large language models (LLMs) toward specific desired outputs. Instead of directly adjusting LLMs, our method employs a small tunable policy model to generate an auxiliary directional stimulus prompt for each input instance.
  arXiv  Detail & Related papers  (2023-02-22T17:44:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.