Robot Planning and Situation Handling with Active Perception

• URL: http://arxiv.org/abs/2604.26988v1
• Date: Tue, 28 Apr 2026 20:56:12 GMT
• Title: Robot Planning and Situation Handling with Active Perception
• Abstract summary: We develop a planning and situation-handling framework, called VAP-TAMP.<n>VAP-TAMP enables robots to actively perceive and address unforeseen situations during plan execution.
• Score: 9.785194821552649
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Current robots are capable of computing plans to accomplish complex tasks. However, real-world environments are inherently open and dynamic, and unforeseen situations frequently arise during plan execution, such as jamming doors and fallen objects on the floor. These situations may result from the robot's own action failures or from external disturbances, such as human activities. Detecting and handling such execution - time situations remains a significant challenge, limiting those robots' ability to achieve long-term autonomy. In this paper, we develop a planning and situation-handling framework, called VAP-TAMP, that enables robots to actively perceive and address unforeseen situations during plan execution. VAP-TAMP leverages action knowledge to strategically prompt vision-language models for active view selection and situation assessment, while constructing and reasoning over scene graphs for integrated task and motion planning. We evaluated VAP-TAMP using service tasks in simulation and on a mobile manipulation platform.

Related papers

• World Action Planner: Generalizable Decision-Making with Action-Conditioned World Models [54.68021681438324]
  We propose World Action Planner, a robot planning system that leverages the reasoning capabilities of Vision-Language Models (VLMs)<n>Our system enables an agent to propose initial action plans and iteratively refine them via optimization and search, reasoning over imagined world model rollouts.<n>We demonstrate that our approach achieves superior performance across compositional tasks, new layouts, and zero-shot generalization scenarios.
  arXiv  Detail & Related papers  (2026-07-30T02:41:52Z)
• Spatially Grounded Long-Horizon Task Planning in the Wild [49.614308474374035]
  GroundedPlanBench is a newly curated benchmark for spatially grounded long-horizon action planning in the wild.<n>Video-to-Spatially Grounded Planning (V2GP) is an automated data generation framework that leverages real-world robot video demonstrations to improve spatially grounded long-horizon planning.
  arXiv  Detail & Related papers  (2026-03-13T05:42:18Z)
• Acting and Planning with Hierarchical Operational Models on a Mobile Robot: A Study with RAE+UPOM [5.758011837296545]
  We present the first physical deployment of an integrated actor-planner system that shares hierarchical operational models for both acting and planning.<n>We implement RAE+UPOM on a mobile manipulator in a real-world deployment for an object collection task.
  arXiv  Detail & Related papers  (2025-07-15T14:20:26Z)
• Beyond Task and Motion Planning: Hierarchical Robot Planning with General-Purpose Policies [32.85839714467011]
  We address the challenge of planning with both kinematic skills and closed-loop motor controllers.<n>We propose a novel method that integrates these controllers into motion planning using Composable Interaction Primitives.
  arXiv  Detail & Related papers  (2025-04-24T19:22:50Z)
• Exploring the Adversarial Vulnerabilities of Vision-Language-Action Models in Robotics [68.36528819227641]
  This paper systematically evaluates the robustness of Vision-Language-Action (VLA) models.<n>We introduce two untargeted attack objectives that leverage spatial foundations to destabilize robotic actions, and a targeted attack objective that manipulates the robotic trajectory.<n>We design an adversarial patch generation approach that places a small, colorful patch within the camera's view, effectively executing the attack in both digital and physical environments.
  arXiv  Detail & Related papers  (2024-11-18T01:52:20Z)
• Grounding Language Models in Autonomous Loco-manipulation Tasks [3.8363685417355557]
  We propose a novel framework that learns, selects, and plans behaviors based on tasks in different scenarios. We leverage the planning and reasoning features of the large language model (LLM), constructing a hierarchical task graph. Experiments in simulation and real-world using the CENTAURO robot show that the language model based planner can efficiently adapt to new loco-manipulation tasks.
  arXiv  Detail & Related papers  (2024-09-02T15:27:48Z)
• Autonomous Behavior Planning For Humanoid Loco-manipulation Through Grounded Language Model [6.9268843428933025]
  Large language models (LLMs) have demonstrated powerful planning and reasoning capabilities for comprehension and processing of semantic information. We propose a novel language-model based framework that enables robots to autonomously plan behaviors and low-level execution under given textual instructions.
  arXiv  Detail & Related papers  (2024-08-15T17:33:32Z)
• A Meta-Engine Framework for Interleaved Task and Motion Planning using Topological Refinements [51.54559117314768]
  Task And Motion Planning (TAMP) is the problem of finding a solution to an automated planning problem. We propose a general and open-source framework for modeling and benchmarking TAMP problems. We introduce an innovative meta-technique to solve TAMP problems involving moving agents and multiple task-state-dependent obstacles.
  arXiv  Detail & Related papers  (2024-08-11T14:57:57Z)
• Commonsense Reasoning for Legged Robot Adaptation with Vision-Language Models [81.55156507635286]
  Legged robots are physically capable of navigating a diverse variety of environments and overcoming a wide range of obstructions. Current learning methods often struggle with generalization to the long tail of unexpected situations without heavy human supervision. We propose a system, VLM-Predictive Control (VLM-PC), combining two key components that we find to be crucial for eliciting on-the-fly, adaptive behavior selection.
  arXiv  Detail & Related papers  (2024-07-02T21:00:30Z)
• Task and Motion Planning for Execution in the Real [24.01204729304763]
  This work generates task and motion plans that include actions cannot be fully grounded at planning time. Execution combines offline planned motions and online behaviors till reaching the task goal. Forty real-robot trials and motivating demonstrations are performed to evaluate the proposed framework. Results show faster execution time, less number of actions, and more success in problems where diverse gaps arise.
  arXiv  Detail & Related papers  (2024-06-05T22:30:40Z)
• Long-Horizon Planning and Execution with Functional Object-Oriented Networks [79.94575713911189]
  We introduce the idea of exploiting object-level knowledge as a FOON for task planning and execution. Our approach automatically transforms FOON into PDDL and leverages off-the-shelf planners, action contexts, and robot skills. We demonstrate our approach on long-horizon tasks in CoppeliaSim and show how learned action contexts can be extended to never-before-seen scenarios.
  arXiv  Detail & Related papers  (2022-07-12T19:29:35Z)
• iCORPP: Interleaved Commonsense Reasoning and Probabilistic Planning on Robots [46.13039152809055]
  We present a novel algorithm, called iCORPP, to simultaneously estimate the current world state, reason about world dynamics, and construct task-oriented controllers. Results show significant improvements in scalability, efficiency, and adaptiveness, compared to competitive baselines.
  arXiv  Detail & Related papers  (2020-04-18T17:46:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.