Related papers: GridToPix: Training Embodied Agents with Minimal Supervision

Related papers

Learning Process Rewards via Success Visitation Matching for Efficient RL [54.67547527674048]
We propose a simple approach to transform a sparse outcome reward into a dense process reward.<n>Our approach relies on training a discriminator to distinguish between previous successful and unsuccessful episodes.<n>By incentivizing the policy to match the visitations over all states, not just those that correspond to task success, this reward provides dense feedback on whether progress is being made towards task completion.
arXiv Detail & Related papers (2026-06-22T17:30:24Z)
RoboReward: General-Purpose Vision-Language Reward Models for Robotics [124.34685604054312]
Vision-language models (VLMs) have shown promise as automatic reward models, yet their effectiveness on real robot tasks is poorly understood.<n>We aim to close this gap by introducing RoboReward, a robotics reward dataset and benchmark built on large-scale real-robot corpora.
arXiv Detail & Related papers (2026-01-02T12:47:34Z)
The Image as Its Own Reward: Reinforcement Learning with Adversarial Reward for Image Generation [52.648073272395635]
We introduce Adv-GRPO, an RL framework with an adversarial reward that iteratively updates both the reward model and the generator.<n>Unlike KL regularization that constrains parameter updates, our learned reward directly guides the generator through its visual outputs.<n>In human evaluation, our method outperforms Flow-GRPO and SD3, achieving 70.0% and 72.4% win rates in image quality and aesthetics, respectively.
arXiv Detail & Related papers (2025-11-25T12:35:57Z)
Game-TARS: Pretrained Foundation Models for Scalable Generalist Multimodal Game Agents [56.25101378553328]
We present Game-TARS, a generalist game agent trained with a unified, scalable action space anchored to human-aligned keyboard-mouse inputs.<n>Game-TARS is pre-trained on over 500B tokens with diverse trajectories and multimodal data.<n> Experiments show that Game-TARS achieves about 2 times the success rate over the previous sota model on open-world Minecraft tasks.
arXiv Detail & Related papers (2025-10-27T17:43:51Z)
Agentic Reinforcement Learning with Implicit Step Rewards [92.26560379363492]
Large language models (LLMs) are increasingly developed as autonomous agents using reinforcement learning (agentic RL)<n>We introduce implicit step rewards for agentic RL (iStar), a general credit-assignment strategy that integrates seamlessly with standard RL algorithms.<n>We evaluate our method on three challenging agent benchmarks, including WebShop and VisualSokoban, as well as open-ended social interactions with unverifiable rewards in SOTOPIA.
arXiv Detail & Related papers (2025-09-23T16:15:42Z)
UI-S1: Advancing GUI Automation via Semi-online Reinforcement Learning [78.86567400365392]
We present Semi-online Reinforcement Learning, a novel paradigm that simulates online RL on offline trajectories.<n>To capture long-term training signals, Semi-online RL introduces discounted future returns into the reward computation.<n>Experiments show that ours Semi-online RL achieves SOTA performance among 7B models across four dynamic benchmarks.
arXiv Detail & Related papers (2025-09-15T03:24:08Z)
Minding Motivation: The Effect of Intrinsic Motivation on Agent Behaviors [2.2169849640518153]
Intrinsic Motivation(IM) methods are an effective solution to reward-sparsity.<n>IM also causes an issue known as reward hacking' where the agent optimize for the new reward at the expense of properly playing the game.<n>This study takes a first step by empirically evaluating the impact on behavior of three IM techniques on the MiniGrid game-like environment.
arXiv Detail & Related papers (2025-07-26T00:49:25Z)
Residual Reward Models for Preference-based Reinforcement Learning [11.797520525358564]
Preference-based Reinforcement Learning (PbRL) provides a way to learn high-performance policies in environments where the reward signal is hard to specify.<n>PbRL can suffer from slow convergence speed since it requires training in a reward model.<n>We propose a method to effectively leverage prior knowledge with a Residual Reward Model (RRM)
arXiv Detail & Related papers (2025-07-01T09:43:57Z)
CaRL: Learning Scalable Planning Policies with Simple Rewards [24.472861976593514]
reinforcement learning is scalable and does not suffer from compounding errors like imitation learning. We show that PPO fails to optimize a popular version of these rewards when the mini-batch size is increased. We propose a new reward design based primarily on optimizing a single intuitive reward term: route completion. We find that PPO scales well with higher mini-batch sizes when trained with our simple reward, even improving performance.
arXiv Detail & Related papers (2025-04-24T17:56:01Z)
VARP: Reinforcement Learning from Vision-Language Model Feedback with Agent Regularized Preferences [13.337649128532307]
Preference-based RL mitigates some of these pitfalls by learning rewards from comparative feedback. A single final-state image generally fails to capture the agent's full motion. We present a two-part solution that both improves feedback accuracy and better aligns reward learning with the agent's policy.
arXiv Detail & Related papers (2025-03-18T01:51:27Z)
Reward Shaping to Mitigate Reward Hacking in RLHF [47.71454266800376]
Reinforcement Learning from Human Feedback (RLHF) is essential for aligning large language models with human values. reward shaping helps stabilize RLHF and partially mitigate reward hacking. We present a comprehensive study of the prevalent reward shaping methods. We propose Preference As Reward (PAR), a novel approach that leverages the latent preferences embedded within the reward model itself as the signal for reinforcement learning.
arXiv Detail & Related papers (2025-02-26T02:57:59Z)
Process Reinforcement through Implicit Rewards [95.7442934212076]
Dense process rewards have proven a more effective alternative to the sparse outcome-level rewards in the inference-time scaling of large language models (LLMs) Dense rewards also offer an appealing choice for the reinforcement learning (RL) of LLMs since their fine-grained rewards have the potential to address some inherent issues of outcome rewards. This can be primarily attributed to the challenges of training process reward models (PRMs) online, where collecting high-quality process labels is prohibitively expensive. We propose PRIME, which enables online PRM updates using only policy rollouts and outcome labels through implict process rewards
arXiv Detail & Related papers (2025-02-03T15:43:48Z)
On-Robot Reinforcement Learning with Goal-Contrastive Rewards [24.415607337006968]
Reinforcement Learning (RL) has the potential to enable robots to learn from their own actions in the real world. We propose GCR (Goal-intensiveive Rewards), a dense reward function learning method that can be trained on passive video demonstrations. GCR combines two loss functions, an implicit value loss function that models how the reward increases when traversing a successful trajectory, and a goal-contrastive loss that discriminates between successful and failed trajectories.
arXiv Detail & Related papers (2024-10-25T22:11:54Z)
Sample-Efficient Preference-based Reinforcement Learning with Dynamics Aware Rewards [2.5101508961934837]
Preference-based reinforcement learning (PbRL) aligns a robot behavior with human preferences via a reward function learned from binary feedback over agent behaviors. We show that dynamics-aware reward functions improve the sample efficiency of PbRL by an order of magnitude.
arXiv Detail & Related papers (2024-02-28T01:41:34Z)
SPOC: Imitating Shortest Paths in Simulation Enables Effective Navigation and Manipulation in the Real World [46.02807945490169]
We show that imitating shortest-path planners in simulation produces agents that can proficiently navigate, explore, and manipulate objects in both simulation and in the real world using only RGB sensors (no depth map or GPS coordinates) This surprising result is enabled by our end-to-end, transformer-based, SPOC architecture, powerful visual encoders paired with extensive image augmentation.
arXiv Detail & Related papers (2023-12-05T18:59:45Z)
Eureka: Human-Level Reward Design via Coding Large Language Models [121.91007140014982]
Large Language Models (LLMs) have excelled as high-level semantic planners for sequential decision-making tasks. We present Eureka, a human-level reward design algorithm powered by LLMs. Eureka exploits the remarkable zero-shot generation, code-writing, and in-context improvement capabilities of state-of-the-art LLMs.
arXiv Detail & Related papers (2023-10-19T17:31:01Z)
Reinforcement Learning with Foundation Priors: Let the Embodied Agent Efficiently Learn on Its Own [59.11934130045106]
We propose Reinforcement Learning with Foundation Priors (RLFP) to utilize guidance and feedback from policy, value, and success-reward foundation models. Within this framework, we introduce the Foundation-guided Actor-Critic (FAC) algorithm, which enables embodied agents to explore more efficiently with automatic reward functions. Our method achieves remarkable performances in various manipulation tasks on both real robots and in simulation.
arXiv Detail & Related papers (2023-10-04T07:56:42Z)
Pixel to policy: DQN Encoders for within & cross-game reinforcement learning [0.0]
Reinforcement Learning can be applied to various tasks, and environments. Many environments have a similar structure, which can be exploited to improve RL performance on other tasks. This work explores as well as compares the performance between RL models being trained from the scratch and on different approaches of transfer learning.
arXiv Detail & Related papers (2023-08-01T06:29:33Z)
Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards [101.7246658985579]
Foundation models are first pre-trained on vast unsupervised datasets and then fine-tuned on labeled data. We propose embracing the heterogeneity of diverse rewards by following a multi-policy strategy. We demonstrate the effectiveness of our approach for text-to-text (summarization, Q&A, helpful assistant, review), text-image (image captioning, text-to-image generation, visual grounding, VQA), and control (locomotion) tasks.
arXiv Detail & Related papers (2023-06-07T14:58:15Z)
A Study on Dense and Sparse (Visual) Rewards in Robot Policy Learning [19.67628391301068]
We study the performance of multiple state-of-the-art deep reinforcement learning algorithms under different types of reward. Our results show that visual dense rewards are more successful than visual sparse rewards and that there is no single best algorithm for all tasks.
arXiv Detail & Related papers (2021-08-06T17:47:48Z)
Combining Off and On-Policy Training in Model-Based Reinforcement Learning [77.34726150561087]
We propose a way to obtain off-policy targets using data from simulated games in MuZero. Our results show that these targets speed up the training process and lead to faster convergence and higher rewards.
arXiv Detail & Related papers (2021-02-24T10:47:26Z)
Semi-supervised reward learning for offline reinforcement learning [71.6909757718301]
Training agents usually requires reward functions, but rewards are seldom available in practice and their engineering is challenging and laborious. We propose semi-supervised learning algorithms that learn from limited annotations and incorporate unlabelled data. In our experiments with a simulated robotic arm, we greatly improve upon behavioural cloning and closely approach the performance achieved with ground truth rewards.
arXiv Detail & Related papers (2020-12-12T20:06:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.