Related papers: Ask When It Pays: Cost-Aware Open-Ended Interaction for Instance Goal Navigation

Ask When It Pays: Cost-Aware Open-Ended Interaction for Instance Goal Navigation

URL: http://arxiv.org/abs/2606.03175v2
Date: Wed, 03 Jun 2026 03:34:51 GMT
Title: Ask When It Pays: Cost-Aware Open-Ended Interaction for Instance Goal Navigation
Abstract summary: Instance Goal Navigation (IGN) requires an embodied agent to find a specific instance object among distractors from an under-specified natural-language description.<n>We recast IGN as a cost-sensitive uncertainty-reduction problem, where the agent should ask the question whose answer provides the largest reduction in navigation uncertainty relative to its penalty.
Score: 33.89987823594985
License: http://creativecommons.org/licenses/by/4.0/
Abstract: Instance Goal Navigation (IGN) requires an embodied agent to find a specific object instance among distractors from an under-specified natural-language description. Such ambiguity often cannot be resolved from perception and language alone, making interaction with an oracle a natural mechanism for disambiguation. Prior interactive methods allow oracle queries but treat lightweight clarification and route-level guidance alike, letting agents boost success rate through repeated high-information questions rather than by resolving the underlying ambiguity efficiently. We recast interactive IGN as a cost-sensitive uncertainty-reduction problem, where the agent should ask the question whose answer provides the largest reduction in navigation uncertainty relative to its penalty. To this end, we apply an information-gain analysis on existing navigation corpora to identify which cues reduce navigation uncertainty, yielding a compact set of question types and data-derived weights. However, existing interactive navigation benchmarks do not model the cost of different question types or evaluate how efficiently agents use interaction, making them unsuitable for studying cost-sensitive interaction. Based on this taxonomy, we construct a benchmark for diagnosing interaction behavior and efficiency, together with a Weighted Success Rate metric that penalizes each query by its derived cost. We further propose a zero-shot MLLM navigator that selectively queries at each decision step only when the expected uncertainty reduction justifies the interaction cost.

Related papers

CLAIM: Leading Open-domain Active Clarification of Large Language Models with Uncertainty Measurement [50.91045324738942]
CLAIM is an uncertainty-driven framework for active clarification learning in open-domain settings.<n>We propose an entropy-driven synthetic data generation pipeline that integrates uncertainty estimation with semantic clustering and reasoning-based judgments.<n> Experimental results demonstrate that CLAIM can learn stable and generalizable clarification strategies without relying on manually labeled data.
arXiv Detail & Related papers (2026-08-12T04:27:45Z)
One More Turn, Less Regret: A Regret-Based Multi-Turn Benchmark for LLMs' Clarification Policies [49.067670811577045]
We introduce RegretBench, a benchmark that evaluates clarification as policy behavior rather than isolated question quality.<n>RegretBench measures intent resolution, interaction cost, ineffective clarification, and regret.<n>Our results show that effective clarification requires more than plausible questions.
arXiv Detail & Related papers (2026-07-23T10:22:11Z)
Proactive Instance Navigation with Comparative Judgment for Ambiguous User Queries [24.694719593521594]
We propose Proactive Instance Navigation with Comparative Judgment (ProCompNav)<n>ProCompNav first constructs a candidate pool and then identifies the target through comparative judgment.<n>On CoIN-Bench, ProCompNav improves Success Rate over interactive baselines with the same minimal input and non-interactive baselines with detailed descriptions.
arXiv Detail & Related papers (2026-05-07T13:19:03Z)
Benchmarking Interaction, Beyond Policy: a Reproducible Benchmark for Collaborative Instance Object Navigation [54.6417539730783]
Question-Asking Navigation (QAsk-Nav) is the first reproducible benchmark for Collaborative Instance Object Navigation (CoIN)<n>We develop Light-CoNav, a lightweight unified model for collaborative navigation that is 3x smaller and 70x faster than existing modular methods.
arXiv Detail & Related papers (2026-03-31T21:45:00Z)
Value of Information: A Framework for Human-Agent Communication [34.068772934008244]
Large Language Model (LLM) agents face a fundamental dilemma: user requests are underspecified, yet agents must decide whether to act on incomplete information or interrupt users for clarification.<n>We introduce a decision-theoretic framework that resolves this trade-off through the Value of Information (VoI)<n>We show that VoI consistently matches or exceeds the best manually-tuned baselines, achieving up to 1.36 utility points higher in high-cost settings.
arXiv Detail & Related papers (2026-01-10T03:07:41Z)
InteractComp: Evaluating Search Agents With Ambiguous Queries [36.05005463045869]
We introduce InteractComp, a benchmark designed to evaluate whether search agents can recognize query ambiguity and actively interact to resolve it during search.<n> Evaluation of 17 models reveals striking failure: the best model achieves only 13.73% accuracy despite 71.50% with complete context.<n>This stagnation, coupled with the immediate feedback inherent to search tasks, makes InteractComp a valuable resource for both evaluating and training interaction capabilities in search agents.
arXiv Detail & Related papers (2025-10-28T17:35:54Z)
Semantic Caching for Low-Cost LLM Serving: From Offline Learning to Online Adaptation [54.61034867177997]
Caching inference responses allows them to be retrieved without another forward pass through the Large Language Models.<n>Traditional exact-match caching overlooks the semantic similarity between queries, leading to unnecessary recomputation.<n>We present a principled, learning-based framework for semantic cache eviction under unknown query and cost distributions.
arXiv Detail & Related papers (2025-08-11T06:53:27Z)
Interactive Agents to Overcome Ambiguity in Software Engineering [61.40183840499932]
AI agents are increasingly being deployed to automate tasks, often based on ambiguous and underspecified user instructions.<n>Making unwarranted assumptions and failing to ask clarifying questions can lead to suboptimal outcomes.<n>We study the ability of LLM agents to handle ambiguous instructions in interactive code generation settings by evaluating proprietary and open-weight models on their performance.
arXiv Detail & Related papers (2025-02-18T17:12:26Z)
SUGAR: Leveraging Contextual Confidence for Smarter Retrieval [28.552283701883766]
We introduce Semantic Uncertainty Guided Adaptive Retrieval (SUGAR)<n>We leverage context-based entropy to actively decide whether to retrieve and to further determine between single-step and multi-step retrieval.<n>Our empirical results show that selective retrieval guided by semantic uncertainty estimation improves the performance across diverse question answering tasks, as well as achieves a more efficient inference.
arXiv Detail & Related papers (2025-01-09T01:24:59Z)
Collaborative Instance Object Navigation: Leveraging Uncertainty-Awareness to Minimize Human-Agent Dialogues [54.81155589931697]
Collaborative Instance object Navigation (CoIN) is a new task setting where the agent actively resolve uncertainties about the target instance.<n>We propose a novel training-free method, Agent-user Interaction with UncerTainty Awareness (AIUTA)<n>First, upon object detection, a Self-Questioner model initiates a self-dialogue within the agent to obtain a complete and accurate observation description.<n>An Interaction Trigger module determines whether to ask a question to the human, continue or halt navigation.
arXiv Detail & Related papers (2024-12-02T08:16:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.