Progress Reward Modeling for Robotic Learning: A Comprehensive Survey

• URL: http://arxiv.org/abs/2607.21655v1
• Date: Wed, 22 Jul 2026 19:29:52 GMT
• Title: Progress Reward Modeling for Robotic Learning: A Comprehensive Survey
• Abstract summary: A terminal success signal only tells the robot whether the task is completed.<n>Recent studies have increasingly explored progress rewards that provide feedback during task execution.<n>Existing methods use different observations, goal specifications, output signals, supervision sources, and evaluation protocols.<n>This makes it difficult to compare them and understand what their results actually validate.
• Score: 18.0494714278744
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robotic learning takes place in dynamic environments with large behavior spaces. A terminal success signal only tells the robot whether the task is completed. It does not explain whether the current behavior is making progress, remaining unchanged, or undoing earlier progress. For this reason, recent studies have increasingly explored progress rewards that provide feedback during task execution. However, the current literature lacks a shared framework. Existing methods use different observations, goal specifications, output signals, supervision sources, and evaluation protocols. This makes it difficult to compare them and understand what their results actually validate. In this survey, we provide a unified view of progress reward modeling for robotic learning. We organize the field in three connected steps. We first study the interface of a progress model. This defines the problem from the outside by asking what information the model receives and what form of progress signal it produces. We then move inside the model and study the methods used to construct this signal. This reveals the different assumptions and mechanisms behind progress estimation and reward generation. Finally, we examine the data and benchmarks that support these methods. This shows how progress supervision is obtained and what different evaluations actually measure. Together, these three perspectives connect what a progress model is, how it is built, and how its quality is validated. We further summarize the main limitations of current approaches and discuss future research directions.

Related papers

• World Models for Embodied Intelligence: From Plausible to Controllable to Actionable [100.13060753774657]
  World models connect perception and decision-making in embodied intelligence by maintaining hidden state, anticipating consequences, comparing interventions, and adapting when execution departs from expectations.<n>This raises a central question: which predictive capabilities improve behavior?<n>We introduce three progressively stronger capability levels: Plausible models preserve task-relevant temporal, geometric, or physical structure; Controllable models additionally predict how interventions alter that structure; and Actionable models translate predictions into measurable gains in planning, action, learning, evaluation, verification, recovery, or data selection.
  arXiv  Detail & Related papers  (2026-09-15T06:22:42Z)
• Monkey See, Can Monkey Do? A Benchmark for Evaluating Robot Skill Learning by Observation [6.576658827863041]
  Learning from Observation (LfO) is a fundamental robotic capability that replicates how humans and animals socially learn from each other.<n>Recent work has demonstrated promising results in learning manipulation skills from human videos.<n>We introduce RoboReel: a unified benchmark for evaluating models that learn policies from human videos.
  arXiv  Detail & Related papers  (2026-09-08T03:48:44Z)
• Dream2Reward: Transition-Alignment Reward Models from Positive Demonstrations for Robotic Manipulation [51.43645327777772]
  We introduce Dream2Reward, which learns a language-conditioned successful latent transition field from positive demonstrations.<n>It penalizes wrong-direction, overshooting, and stagnant motion even when the resulting observation appears to show progress.<n>Across online and offline policy learning, the same frozen reward model reduces reward hacking and supports stronger downstream performance.
  arXiv  Detail & Related papers  (2026-08-19T10:44:20Z)
• Can In-Context Learning Support Intrinsic Curiosity? [25.43664318918942]
  We investigate whether the emergent in-context learning capabilities of sequence models can eliminate this bottleneck.<n>Specifically, we evaluate whether an exploration policy can be trained to maximize learning progress.<n>We prove a positive result for a broad subclass of non-temporal settings.
  arXiv  Detail & Related papers  (2026-06-17T18:11:06Z)
• ProgressVLA: Progress-Guided Diffusion Policy for Vision-Language Robotic Manipulation [57.07494675832939]
  Most existing vision-language-action (VLA) models for robotic manipulation lack progress awareness.<n>In this work, we investigate the estimation and integration of task progress, proposing a novel model named textbf vla<n>Experiments on the CALVIN and LIBERO benchmarks, alongside real-world robot deployment, consistently demonstrate substantial improvements in success rates.
  arXiv  Detail & Related papers  (2026-03-29T12:38:11Z)
• PROGRESSLM: Towards Progress Reasoning in Vision-Language Models [10.481670664271073]
  Estimating task progress requires reasoning over long-horizon dynamics rather than recognizing static visual content.<n>We introduce Progress-Bench, a benchmark for systematically evaluating progress reasoning in Vision-Language Models.<n>We further explore a human-inspired two-stage progress reasoning paradigm through both training-free prompting and training-based approach.
  arXiv  Detail & Related papers  (2026-01-21T17:56:59Z)
• Playpen: An Environment for Exploring Learning Through Conversational Interaction [84.0413820245725]
  We investigate whether Dialogue Games can also serve as a source of feedback signals for learning.<n>We introduce Playpen, an environment for off- and online learning through Dialogue Game self-play.<n>We find that imitation learning through SFT improves performance on unseen instances, but negatively impacts other skills.
  arXiv  Detail & Related papers  (2025-04-11T14:49:33Z)
• How Much Progress Did I Make? An Unexplored Human Feedback Signal for Teaching Robots [6.288170490755676]
  We characterize and test a little-used teaching signal: textitprogress, designed to represent the completion percentage of a task. We find that progress indicates whether the task is successfully performed, reflects the degree of task completion, identifies unproductive but harmless behaviors, and is likely to be more consistent across participants. An additional contribution of our work is a dataset of 40 non-expert demonstrations from the public space study through an ice cream topping-adding task.
  arXiv  Detail & Related papers  (2024-07-08T23:47:13Z)
• Is there progress in activity progress prediction? [19.841311026235875]
  We find that current progress prediction methods seem not to extract useful visual information for the progress prediction task. We design a precisely controlled dataset for activity progress prediction and on this synthetic dataset we show that the considered methods can make use of the visual information. We conclude that the progress prediction task is ill-posed on the currently used real-world datasets.
  arXiv  Detail & Related papers  (2023-08-10T12:23:47Z)
• Don't Copy the Teacher: Data and Model Challenges in Embodied Dialogue [92.01165203498299]
  Embodied dialogue instruction following requires an agent to complete a complex sequence of tasks from a natural language exchange. This paper argues that imitation learning (IL) and related low-level metrics are actually misleading and do not align with the goals of embodied dialogue research.
  arXiv  Detail & Related papers  (2022-10-10T05:51:40Z)
• Model-Based Visual Planning with Self-Supervised Functional Distances [104.83979811803466]
  We present a self-supervised method for model-based visual goal reaching. Our approach learns entirely using offline, unlabeled data. We find that this approach substantially outperforms both model-free and model-based prior methods.
  arXiv  Detail & Related papers  (2020-12-30T23:59:09Z)
• Value-driven Hindsight Modelling [68.658900923595]
  Value estimation is a critical component of the reinforcement learning (RL) paradigm. Model learning can make use of the rich transition structure present in sequences of observations, but this approach is usually not sensitive to the reward function. We develop an approach for representation learning in RL that sits in between these two extremes. This provides tractable prediction targets that are directly relevant for a task, and can thus accelerate learning the value function.
  arXiv  Detail & Related papers  (2020-02-19T18:10:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.