When Should a Robot Think? Resource-Aware Reasoning via Reinforcement Learning for Embodied Robotic Decision-Making

• URL: http://arxiv.org/abs/2603.16673v1
• Date: Tue, 17 Mar 2026 15:38:50 GMT
• Title: When Should a Robot Think? Resource-Aware Reasoning via Reinforcement Learning for Embodied Robotic Decision-Making
• Abstract summary: Embodied robotic systems increasingly rely on large language model (LLM)-based agents to support high-level reasoning.<n>This raises a fundamental question for embodied agents: when should the agent reason, and when should it act?<n>We propose RARRL (Resource-Aware Reasoning via Reinforcement Learning), a hierarchical framework for resource-aware orchestration of embodied agents.
• Score: 68.12864562049957
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Embodied robotic systems increasingly rely on large language model (LLM)-based agents to support high-level reasoning, planning, and decision-making during interactions with the environment. However, invoking LLM reasoning introduces substantial computational latency and resource overhead, which can interrupt action execution and reduce system reliability. Excessive reasoning may delay actions, while insufficient reasoning often leads to incorrect decisions and task failures. This raises a fundamental question for embodied agents: when should the agent reason, and when should it act? In this work, we propose RARRL (Resource-Aware Reasoning via Reinforcement Learning), a hierarchical framework for resource-aware orchestration of embodied agents. Rather than learning low-level control policies, RARRL learns a high-level orchestration policy that operates at the agent's decision-making layer. This policy enables the agent to adaptively determine whether to invoke reasoning, which reasoning role to employ, and how much computational budget to allocate based on current observations, execution history, and remaining resources. Extensive experiments, including evaluations with empirical latency profiles derived from the ALFRED benchmark, show that RARRL consistently improves task success rates while reducing execution latency and enhancing robustness compared with fixed or heuristic reasoning strategies. These results demonstrate that adaptive reasoning control is essential for building reliable and efficient embodied robotic agents.

Related papers

• Don't Overthink, Don't Underthink: Toward Adaptive Reasoning in Agentic AI [0.15039745292757667]
  Major challenge for next-generation agentic AI is not merely how much reasoning a language model should perform, but how it should allocate reasoning according to evolving task demands.<n>We characterize over-reasoning and under-reasoning as recurring failure modes of misallocated reasoning and evaluate them on MATH-500 and the GAIA public validation benchmark.
  arXiv  Detail & Related papers  (2026-08-26T22:45:36Z)
• On-Device Robotic Planning: Eliminating Inference Redundancy for Efficient Decision-Making [4.723358389541955]
  Reasoning-based robotic policies using large language and vision-language models achieve strong semantic planning capabilities but mostly suffer from a high inference latency.<n>We present REIS, a human cognition inspired robotic decision-making framework that minimizes unnecessary reasoning while preserving semantic adaptability.
  arXiv  Detail & Related papers  (2026-05-29T15:51:42Z)
• Harnessing LLM Agents with Skill Programs [58.356514745548026]
  We introduce HASP, a new framework that upgrades skills into executable Program Functions (PFs)<n>PFs act as executable guardrails that activate on failure-prone states and modify the next action or inject corrective context.<n> HASP drives substantial gains compared to both training-free and training-based methods on web-search, math reasoning, and coding tasks.
  arXiv  Detail & Related papers  (2026-05-18T01:35:11Z)
• AIPO: Learning to Reason from Active Interaction [54.10819421625103]
  AIPO enables the policy model to proactively consult three functional collaborative agents when encountering reasoning bottlenecks.<n>AIPO consistently improves reasoning performance, generalizes robustly across different policy models and RLVR algorithms, and effectively expands the reasoning capability boundary of the policy model.
  arXiv  Detail & Related papers  (2026-05-08T19:06:55Z)
• Position: agentic AI orchestration should be Bayes-consistent [92.04373485710285]
  This paper argues that coherent decision-making requires Bayesian principles at the orchestration level of the agentic system.<n>It provides examples and design patterns to illustrate how beliefs and utility-aware policies can improve agentic AI orchestration.
  arXiv  Detail & Related papers  (2026-05-01T15:43:43Z)
• AgentCollab: A Self-Evaluation-Driven Collaboration Paradigm for Efficient LLM Agents [37.232397795331444]
  We present AgentCollab, a self-driven collaborative inference framework that dynamically coordinates models with different reasoning capacities during agent execution.<n>Instead of relying on external routing modules, the framework uses the agent's own self-reflection signal to determine whether the current reasoning trajectory is making meaningful progress.
  arXiv  Detail & Related papers  (2026-03-27T03:07:34Z)
• Utility-Guided Agent Orchestration for Efficient LLM Tool Use [24.97962970486756]
  We study agent orchestration as an explicit decision problem.<n>We propose a utility-guided orchestration policy that selects among actions such as respond, retrieve, tool call, verify, and stop.
  arXiv  Detail & Related papers  (2026-03-20T12:29:12Z)
• RISER: Orchestrating Latent Reasoning Skills for Adaptive Activation Steering [62.63376387138257]
  We propose a plug-and-play intervention framework that adaptively steers large language models (LLMs) reasoning in activation space.<n>RISER constructs a library of reusable reasoning vectors and employs a lightweight Router to dynamically compose them for each input.<n>The Router is optimized via reinforcement learning under task-level rewards, activating latent cognitive primitives in an emergent and compositional manner.
  arXiv  Detail & Related papers  (2026-01-14T08:04:33Z)
• The Cost of Dynamic Reasoning: Demystifying AI Agents and Test-Time Scaling from an AI Infrastructure Perspective [3.0868637098088403]
  Large-language-model (LLM)-based AI agents have recently showcased impressive versatility by employing dynamic reasoning.<n>This paper presents the first comprehensive system-level analysis of AI agents, quantifying their resource usage, latency behavior, energy consumption, and test-time scaling strategies.<n>Our findings reveal that while agents improve accuracy with increased compute, they suffer from rapidly diminishing returns, widening latency variance, and unsustainable infrastructure costs.
  arXiv  Detail & Related papers  (2025-06-04T14:37:54Z)
• The Real Barrier to LLM Agent Usability is Agentic ROI [110.31127571114635]
  Large Language Model (LLM) agents represent a promising shift in human-AI interaction.<n>We highlight a critical usability gap in high-demand, mass-market applications.
  arXiv  Detail & Related papers  (2025-05-23T11:40:58Z)
• REBEL: Reward Regularization-Based Approach for Robotic Reinforcement Learning from Human Feedback [61.54791065013767]
  A misalignment between the reward function and human preferences can lead to catastrophic outcomes in the real world.<n>Recent methods aim to mitigate misalignment by learning reward functions from human preferences.<n>We propose a novel concept of reward regularization within the robotic RLHF framework.
  arXiv  Detail & Related papers  (2023-12-22T04:56:37Z)
• Decision-Making Among Bounded Rational Agents [5.24482648010213]
  We introduce the concept of bounded rationality from an information-theoretic view into the game-theoretic framework. This allows the robots to reason other agents' sub-optimal behaviors and act accordingly under their computational constraints. We demonstrate that the resulting framework allows the robots to reason about different levels of rational behaviors of other agents and compute a reasonable strategy under its computational constraint.
  arXiv  Detail & Related papers  (2022-10-17T00:29:24Z)
• Accelerated Reinforcement Learning for Temporal Logic Control Objectives [10.216293366496688]
  This paper addresses the problem of learning control policies for mobile robots modeled as unknown Markov Decision Processes (MDPs) We propose a novel accelerated model-based reinforcement learning (RL) algorithm for control objectives that is capable of learning control policies significantly faster than related approaches.
  arXiv  Detail & Related papers  (2022-05-09T17:09:51Z)
• Automated Machine Learning, Bounded Rationality, and Rational Metareasoning [62.997667081978825]
  We will look at automated machine learning (AutoML) and related problems from the perspective of bounded rationality. Taking actions under bounded resources requires an agent to reflect on how to use these resources in an optimal way.
  arXiv  Detail & Related papers  (2021-09-10T09:10:20Z)
• Enhancing reinforcement learning by a finite reward response filter with a case study in intelligent structural control [0.0]
  In many reinforcement learning (RL) problems, it takes some time until a taken action by the agent reaches its maximum effect on the environment. This paper introduces an applicable enhanced Q-learning method in which at the beginning of the learning phase, the agent takes a single action. We have applied the developed method to a structural control problem in which the goal of the agent is to reduce the vibrations of a building subjected to earthquake excitations with a specified delay.
  arXiv  Detail & Related papers  (2020-10-25T19:28:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.