Beyond Pixels: Learning Invariant Rewards for Real-World Robotics From a Few Demonstrations

• URL: http://arxiv.org/abs/2605.22123v1
• Date: Thu, 21 May 2026 07:55:35 GMT
• Title: Beyond Pixels: Learning Invariant Rewards for Real-World Robotics From a Few Demonstrations
• Abstract summary: In open-world manipulation problems, a single task can appear in numerous variants through different object instances, positions, and camera viewpoints.<n>Recent vision-based reward models tend to memorize specific pixel distributions and fail to generalize beyond their training conditions.<n>We propose a framework that learns invariant symbolic reward functions from as few as five demonstrations.
• Score: 15.30585150107243
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Designing reward functions that generalize beyond controlled laboratory settings remains a fundamental challenge in reinforcement learning for robotics. In open-world manipulation problems, a single task can appear in numerous variants through different object instances, positions, and camera viewpoints. Recent vision-based reward models tend to memorize specific pixel distributions and fail to generalize beyond their training conditions. To address this, we propose a framework that learns invariant symbolic reward functions from as few as five demonstrations. The insight is to shift from visual feature-fitting to the discovery of behavioral invariants: task-level properties that remain constant across diverse visual instantiations. The framework has two coupled components: a structural reward formulation that encodes task-level strategies and physical constraints while preserving optimal policy invariance, and a hybrid symbolic-numerical procedure that distills these invariants from demonstrations without online interaction. Experiments on eight Meta-World tasks and three Franka manipulation tasks demonstrate that our method achieves stronger process alignment and policy rollout ranking abilities compared to baselines, accelerating downstream policy learning. Three real-world out-of-distribution experiments further show that the same learned reward generalizes zero-shot to position, viewpoint, and object variations, enabling a single reward representation to be reused across diverse task variants in practice.

Related papers

• Task Editing for Generalizable 3D Visuomotor Policy Learning [50.590696755853365]
  3D visuomotor policies offer a promising direction for complex robotic manipulation, as depth maps and point clouds provide rich geometric information for spatial reasoning.<n>Existing methods commonly use demonstration generation strategies to improve data efficiency by applying object-centric transformations to human-collected demonstrations.<n>We propose Task-Edit, a novel demonstration generation framework that generates diverse trajectories from a task-centric editing perspective.
  arXiv  Detail & Related papers  (2026-06-05T07:54:49Z)
• Multi-Camera View Scaling for Data-Efficient Robot Imitation Learning [57.88781687419521]
  Multiple synchronized camera perspectives are used to generate pseudo-demonstrations from each expert trajectory.<n>Experiments in simulation and real-world manipulation tasks demonstrate significant gains in data efficiency and generalization.<n>Our results suggest that scaling camera views provides a practical and scalable solution for imitation learning.
  arXiv  Detail & Related papers  (2026-04-01T07:00:44Z)
• Learning to Act Robustly with View-Invariant Latent Actions [8.446887947386559]
  Vision-based robotic policies often struggle with even minor viewpoint changes, underscoring the need for view-invariant visual representations.<n>We propose View-Invariant Latent Action (VILA), which models a latent action capturing transition patterns across trajectories to learn view-invariant representations grounded in physical dynamics.<n> Experiments in both simulation and the real world show that VILA-based policies generalize effectively to unseen viewpoints and transfer well to new tasks.
  arXiv  Detail & Related papers  (2026-01-06T13:14:01Z)
• CORE: Code-based Inverse Self-Training Framework with Graph Expansion for Virtual Agents [69.88668127604875]
  We present CORE, a Code-based Inverse Self-Training Framework with Graph Expansion.<n> CORE bridges imitation and exploration, offering a novel training framework that promotes behavioral diversity.<n> Experiments on Web and Android platforms demonstrate that CORE significantly improves both overall performance and generalization.
  arXiv  Detail & Related papers  (2026-01-05T15:24:05Z)
• Zero-Shot Visual Generalization in Robot Manipulation [0.13280779791485384]
  Current approaches often sidestep the problem by relying on invariant representations such as point clouds and depth.<n>Disentangled representation learning has recently shown promise in enabling vision-based reinforcement learning policies to be robust to visual distribution shifts.<n>We demonstrate zero-shot adaptability to visual perturbations in both simulation and on real hardware.
  arXiv  Detail & Related papers  (2025-05-16T22:01:46Z)
• Subtask-Aware Visual Reward Learning from Segmented Demonstrations [97.80917991633248]
  This paper introduces REDS: REward learning from Demonstration with Demonstrations, a novel reward learning framework.<n>We train a dense reward function conditioned on video segments and their corresponding subtasks to ensure alignment with ground-truth reward signals.<n>Our experiments show that REDS significantly outperforms baseline methods on complex robotic manipulation tasks in Meta-World.
  arXiv  Detail & Related papers  (2025-02-28T01:25:37Z)
• A Real-to-Sim-to-Real Approach to Robotic Manipulation with VLM-Generated Iterative Keypoint Rewards [29.923942622540356]
  We introduce Iterative Keypoint Reward (IKER), a Python-based reward function that serves as a dynamic task specification.<n>We reconstruct real-world scenes in simulation and use the generated rewards to train reinforcement learning policies.<n>The results highlight IKER's effectiveness in enabling robots to perform multi-step tasks in dynamic environments.
  arXiv  Detail & Related papers  (2025-02-12T18:57:22Z)
• Learning Generalizable 3D Manipulation With 10 Demonstrations [16.502781729164973]
  We present a novel framework that learns manipulation skills from as few as 10 demonstrations. We validate our framework through extensive experiments on both simulation benchmarks and real-world robotic systems. This work shows significant potential for advancing efficient, generalizable manipulation skill learning in real-world applications.
  arXiv  Detail & Related papers  (2024-11-15T14:01:02Z)
• One-shot Imitation in a Non-Stationary Environment via Multi-Modal Skill [6.294766893350108]
  We present a skill-based imitation learning framework enabling one-shot imitation and zero-shot adaptation. We leverage a vision-language model to learn a semantic skill set from offline video datasets. We evaluate our framework with various one-shot imitation scenarios for extended multi-stage Meta-world tasks.
  arXiv  Detail & Related papers  (2024-02-13T11:01:52Z)
• Leveraging sparse and shared feature activations for disentangled representation learning [112.22699167017471]
  We propose to leverage knowledge extracted from a diversified set of supervised tasks to learn a common disentangled representation. We validate our approach on six real world distribution shift benchmarks, and different data modalities.
  arXiv  Detail & Related papers  (2023-04-17T01:33:24Z)
• Plug and Play, Model-Based Reinforcement Learning [60.813074750879615]
  We introduce an object-based representation that allows zero-shot integration of new objects from known object classes. This is achieved by representing the global transition dynamics as a union of local transition functions. Experiments show that our representation can achieve sample-efficiency in a variety of set-ups.
  arXiv  Detail & Related papers  (2021-08-20T01:20:15Z)
• Self-Supervised Representation Learning from Flow Equivariance [97.13056332559526]
  We present a new self-supervised learning representation framework that can be directly deployed on a video stream of complex scenes. Our representations, learned from high-resolution raw video, can be readily used for downstream tasks on static images.
  arXiv  Detail & Related papers  (2021-01-16T23:44:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.