Robot Critics that Sweat the Small Stuff

• URL: http://arxiv.org/abs/2606.21572v1
• Date: Fri, 19 Jun 2026 16:14:12 GMT
• Title: Robot Critics that Sweat the Small Stuff
• Abstract summary: We introduce a method to fine-tune critics by constructing pairwise progress supervision using success and failure rollouts obtained from a policy.<n>Our critic excels at fine-grained progress reasoning and subtle failure detection, outperforming prior progress reasoning baselines.
• Score: 44.13358184052461
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large vision-language models contain several priors about the world and object interactions, making them useful critics during inference to steer robot policies towards success. However, closed-loop robot manipulation requires judging small visual differences between success and failure, which remains a challenge for current VLMs. We introduce a method to fine-tune critics by constructing pairwise progress supervision using success and failure rollouts obtained from a policy. Our fine-tuned critic excels at fine-grained progress reasoning and subtle failure detection, outperforming prior progress reasoning baselines. Additionally, we use an action-conditioned video model to predict the visual effect of several candidate actions sampled from a policy, and show that our critic can correctly identify successful candidates to execute, improving the average policy success rate by 11% across real-world tasks and 5.9% across simulation tasks.

Related papers

• DreamAvoid: Critical-Phase Test-Time Dreaming to Avoid Failures in VLA Policies [65.27025563507961]
  We propose DreamAvoid, a critical-phase test-time dreaming framework for Vision-Language-Action (VLA) models.<n>We also introduce an autonomous boundary learning paradigm to refine the system's understanding of the subtle boundary between success and failure.<n>The results demonstrate that DreamAvoid can effectively avoid failures, thereby improving the overall task success rate.
  arXiv  Detail & Related papers  (2026-05-12T08:27:16Z)
• Asymmetric Actor-Critic for Multi-turn LLM Agents [50.245019205783855]
  We propose an asymmetric actor-critic framework for reliable conversational agents.<n>A powerful proprietary LLM acts as the actor, while a smaller open-source critic provides runtime supervision.<n>We show that our approach significantly improves reliability and task success over strong single-agent baselines.
  arXiv  Detail & Related papers  (2026-03-31T22:56:21Z)
• Update-Free On-Policy Steering via Verifiers [47.56134859358476]
  Behavior Cloning (BC) has become one of the most prevalent methods for enabling robots to mimic human demonstrations.<n>We propose UF-OPS, an Update-Free On-Policy Steering method that enables the robot to predict the success likelihood of its actions.<n>We present results from both simulation and real-world data and achieve an average 49% improvement in success rate over the base policy across 5 real tasks.
  arXiv  Detail & Related papers  (2026-03-10T23:55:52Z)
• Impact of Different Failures on a Robot's Perceived Reliability [9.607560620828718]
  We examine how various kinds of failures affect the robot's perceived reliability (PR)<n>We show that successes immediately following a failure have the same effect on PR as successes without a preceding failure.<n>Our findings highlight which robot failures are in higher need of repair in a human-robot interaction, and how trust could be recovered by robot successes.
  arXiv  Detail & Related papers  (2026-03-09T18:27:53Z)
• Verified Critical Step Optimization for LLM Agents [67.05296684575445]
  Critical Step Optimization focuses preference learning on verified critical steps.<n>Method starts from failed policy trajectories rather than expert demonstrations.<n>Experiments on GAIA-Text-103 and XBench-DeepSearch show that CSO achieves 37% and 26% relative improvement over the SFT baseline.
  arXiv  Detail & Related papers  (2026-02-03T11:41:02Z)
• Critique-RL: Training Language Models for Critiquing through Two-Stage Reinforcement Learning [89.60378227969643]
  We propose Critique-RL, an online RL approach for developing critiquing language models without stronger supervision.<n>Our approach operates on a two-player paradigm: the actor generates a response, the critic provides feedback, and the actor refines the response accordingly.<n>Experiments across various tasks and models show that Critique-RL delivers substantial performance improvements.
  arXiv  Detail & Related papers  (2025-10-28T11:37:01Z)
• Evaluating Uncertainty and Quality of Visual Language Action-enabled Robots [13.26825865228582]
  We propose eight uncertainty metrics and five quality metrics specifically designed for VLA models for robotic manipulation tasks.<n>We assess their effectiveness through a large-scale empirical study involving 908 successful task executions from three state-of-the-art VLA models.
  arXiv  Detail & Related papers  (2025-07-22T22:15:59Z)
• RoboEval: Where Robotic Manipulation Meets Structured and Scalable Evaluation [32.080769025457926]
  We present RoboEval, a simulation benchmark and structured evaluation framework designed to reveal the limitations of current bimanual manipulation policies.<n>RoboEval introduces a suite of tiered, semantically grounded tasks that systematically challenge spatial, physical, and coordination capabilities.<n>We find that behavioral metrics correlate with success in over half of task-metric pairs, and remain informative even when binary success saturates.
  arXiv  Detail & Related papers  (2025-07-01T05:33:16Z)
• LLM-Safety Evaluations Lack Robustness [58.334290876531036]
  We argue that current safety alignment research efforts for large language models are hindered by many intertwined sources of noise.<n>We propose a set of guidelines for reducing noise and bias in evaluations of future attack and defense papers.
  arXiv  Detail & Related papers  (2025-03-04T12:55:07Z)
• Task Success is not Enough: Investigating the Use of Video-Language Models as Behavior Critics for Catching Undesirable Agent Behaviors [22.728139463769896]
  Large-scale generative models are useful for sampling meaningful candidate solutions, yet they often overlook task constraints and user preferences. In the context of embodied AI, verification often solely involves assessing whether goal conditions specified in the instructions have been met. Given the scope of robot tasks, it is infeasible to construct scripted verifiers akin to those used for explicit-knowledge tasks like Go. This begs the question: can we use large vision and language models (VLMs), which are approximately omniscient, as scalable Behavior Critics to catch undesirable robot behaviors in videos?
  arXiv  Detail & Related papers  (2024-02-06T18:07:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.