STEAM: Self-Supervised Temporal Ensemble Advantage Modeling for Real-World Robot Learning

• URL: http://arxiv.org/abs/2606.29834v1
• Date: Mon, 29 Jun 2026 06:19:35 GMT
• Title: STEAM: Self-Supervised Temporal Ensemble Advantage Modeling for Real-World Robot Learning
• Abstract summary: Policy learning requires frame-level advantages that distinguish reliable local progress from failures and regressions.<n>We propose Self-supervised Temporal Temporal Ensemble Advantage Modeling (STEAM), a label-free method that learns such advantages from expert demonstrations.
• Score: 76.76959348575764
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Real-world robot learning increasingly relies on heterogeneous data, but demonstrations and rollouts often mix useful progress with stalls, corrections, and suboptimal behavior. Effective policy learning therefore requires frame-level advantages that distinguish reliable local progress from failures and regressions. We propose Self-supervised Temporal Ensemble Advantage Modeling (STEAM), a label-free method that learns such advantages from expert demonstrations. STEAM trains an ensemble of temporal-offset predictors on frame pairs within expert trajectories, using the normalized temporal offset between two frames as a self-supervised signal. Each predictor maps a frame pair to a distribution over temporal offsets, which is converted into a scalar advantage. STEAM then takes the minimum advantage across the ensemble to score mixed-quality rollout data conservatively. Across real-world bimanual towel folding, chip checkout, cola restocking, and single-arm pick-and-place tasks, STEAM identifies stalls, failures, and recoveries. When combined with CFGRL, STEAM further improves policy success rate by 59%, 54.3%, 23% and 16.2% over baselines, respectively.

Related papers

• ST-LoRA: Single Trajectory LoRA Ensemble for Uncertainty Aware Agricultural Segmentation [4.391214332238358]
  Decision-support in digital agriculture requires accurate predictions and well-calibrated uncertainty estimates.<n>We propose ST-LoRA, a parameter-efficient ensemble framework that builds diverse ensemble members from a single training trajectory.<n> ST-LoRA matches or exceeds full-rank ensembles in segmentation accuracy and calibration across both datasets and architectures.
  arXiv  Detail & Related papers  (2026-08-02T22:49:28Z)
• Rank-Then-Act: Reward-Free Control from Frame-Order Progress [5.357451930622806]
  Rank-Then-Act (RTA) is a framework for learning control policies from expert video demonstrations without environment rewards.<n>We train a Vision-Language Model (VLM) offline as a progress-based ordinal scorer.<n>We compute the Spearman rank correlation between predicted progress rankings and true temporal indices, yielding a bounded, scale-invariant learning signal.
  arXiv  Detail & Related papers  (2026-07-02T08:50:32Z)
• Perfect Demo Makes Poor Teacher: Learning Robust Alignment from Critical Motion Segments [53.85970147743299]
  Expert demonstrations are widely assumed to be the gold standard for robot imitation learning.<n>Yet for fine-grained manipulation such as insertion, stacking, and alignment, we uncover a counterintuitive failure mode: fluent demonstrations can be poor teachers.<n>We introduce STAIR, a compact dynamic feature that bridges the vision-language model and the action expert, distilling the short-horizon motion already recorded in each trajectory into dense, motion-aware supervision.
  arXiv  Detail & Related papers  (2026-06-14T04:15:29Z)
• Multimodal Learning on Low-Quality Data with Conformal Predictive Self-Calibration [72.0672328514289]
  Multimodal learning often grapples with the challenge of low-quality data.<n>We propose a unified framework, termed Conformal Predictive Self-Calibration.<n>Our framework consistently outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-05-05T14:48:52Z)
• MindGames Arena Generalization Track: In2AI Solution with Delayed Per-Step Reward Attribution [45.957713330643934]
  Training language model agents for multi-agent strategic interaction presents a core difficulty.<n>Standard reinforcement learning assumes that rewards can be assigned at each step, but this assumption fails in settings where outcomes are entangled across time and agents.<n>We introduce delayed per-step reward attribution with eligibility gating, an episode lifecycle and postprocessing pipeline.
  arXiv  Detail & Related papers  (2026-04-13T23:59:03Z)
• Human-in-the-loop Online Rejection Sampling for Robotic Manipulation [55.99788088622936]
  Hi-ORS stabilizes value estimation by filtering out negatively rewarded samples during online fine-tuning.<n>Hi-ORS fine-tunes a pi-base policy to master contact-rich manipulation in just 1.5 hours of real-world training.
  arXiv  Detail & Related papers  (2025-10-30T11:53:08Z)
• RHYTHM: Reasoning with Hierarchical Temporal Tokenization for Human Mobility [9.200793414310182]
  We introduce RHYTHM (Reasoning with Hierarchical Temporal Tokenization for Human Mobility), a unified framework for predicting human mobility.<n>We use large language models (LLMs) as general-purpose predictors and reasoners.<n> RHYTHM achieves a 2.4% in overall accuracy, a 5.0% increase on weekends, and a 24.6% reduction in training time.
  arXiv  Detail & Related papers  (2025-09-27T04:55:56Z)
• Generative Regression Based Watch Time Prediction for Short-Video Recommendation [36.95095097454143]
  Watch time prediction has emerged as a pivotal task in short video recommendation systems.<n>Recent studies have attempted to address these issues by converting the continuous watch time estimation into an ordinal regression task.<n>We propose a novel Generative Regression (GR) framework that reformulates WTP as a sequence generation task.
  arXiv  Detail & Related papers  (2024-12-28T16:48:55Z)
• Right on Time: Revising Time Series Models by Constraining their Explanations [18.266824309661168]
  Deep time series models often suffer from reliability issues due to their tendency to rely on spurious correlations.<n>To mitigate such shortcuts and prevent "Clever-Hans" moments in time series models, we introduce Right on Time (RioT)<n>RioT constrains the model, steering it away from annotated spurious correlations.
  arXiv  Detail & Related papers  (2024-02-20T11:15:13Z)
• REBAR: Retrieval-Based Reconstruction for Time-series Contrastive Learning [64.08293076551601]
  We propose a novel method of using a learned measure for identifying positive pairs. Our Retrieval-Based Reconstruction measure measures the similarity between two sequences. We show that the REBAR error is a predictor of mutual class membership.
  arXiv  Detail & Related papers  (2023-11-01T13:44:45Z)
• Distributionally Robust Models with Parametric Likelihood Ratios [123.05074253513935]
  Three simple ideas allow us to train models with DRO using a broader class of parametric likelihood ratios. We find that models trained with the resulting parametric adversaries are consistently more robust to subpopulation shifts when compared to other DRO approaches.
  arXiv  Detail & Related papers  (2022-04-13T12:43:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.