Observation-Grounded Self-Predictive Reinforcement Learning for Visual Continuous Control

• URL: http://arxiv.org/abs/2608.05989v1
• Date: Thu, 06 Aug 2026 13:02:59 GMT
• Title: Observation-Grounded Self-Predictive Reinforcement Learning for Visual Continuous Control
• Abstract summary: We propose Observation-Grounded Self-Predictive Representations (OG-SPR) for continuous control.<n>OG-SPR incorporates two core auxiliary objectives: multi-step latent self-prediction and next-observation prediction.<n> Experiments show that OG-SPR improves aggregate performance over state-of-the-art self-predictive and observation-predictive RL methods.
• Score: 10.054153317469096
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sample-efficient policy learning from pixels is a long-standing challenge in reinforcement learning (RL). Recent dynamics-based representation learning methods have significantly improved the sample efficiency of model-free visual RL by learning dynamics-aware representations through auxiliary prediction performed either in latent space (self-prediction) or observation space (observation prediction). However, state-of-the-art methods from both categories still struggle on challenging visual control tasks when training data is limited. We posit that relying on either predictive objective alone may be insufficient. In contrast, observation prediction grounds learned representations in observation-level dynamics, but does not directly regularize the temporal predictability of latent representations over extended horizons. In this paper, we propose Observation-Grounded Self-Predictive Representations (OG-SPR), a model-free visual RL algorithm for continuous control that learns representations that are both temporally predictive in latent space and grounded in observation-level dynamics. OG-SPR incorporates two core auxiliary objectives: multi-step latent self-prediction and next-observation prediction. We empirically show that directly imposing latent self-prediction on the shared representation may over-constrain it and does not necessarily improve performance. To address this issue, OG-SPR introduces two lightweight adapters for latent self-prediction, allowing the shared representation to benefit from temporally predictive signals without being forced to directly satisfy the self-prediction objective. Experiments on 28 visual control tasks from the DeepMind Control Suite show that OG-SPR improves aggregate performance over state-of-the-art self-predictive and observation-predictive RL methods, with particularly pronounced gains in challenging domains such as dog and humanoid.

Related papers

• NASDAQ: Normalized Observation Space Dynamics-Augmented Q-Learning [13.486549278186928]
  Augmenting model-free reinforcement learning (RL) with representations learned through observation dynamics prediction (observation-predictive RL) can improve sample efficiency and performance.<n>We propose a novel normalization method tailored to online RL, which normalizes low-dimensional observations and balances the resulting losses and gradients.<n>We introduce Normalized Observation Space Dynamics-Augmented Q-learning ( NASDAQ), a framework for observation-predictive RL applicable across diverse domains.
  arXiv  Detail & Related papers  (2026-06-19T10:21:04Z)
• ReViP: Reducing False Completion in Vision-Language-Action Models with Vision-Proprioception Rebalance [50.05984919728878]
  We present ReViP, a novel VLA framework with Vision-Proprioception Rebalance to enhance visual grounding and robustness under perturbations.<n>Specifically, we use an external VLM as a task-stage observer to extract real-time task-centric visual cues from visual observations.<n>To evaluate false completion, we propose the first False-Completion Benchmark Suite built on LIBERO with controlled settings such as Object-Drop.
  arXiv  Detail & Related papers  (2026-01-23T11:31:07Z)
• Abstain Mask Retain Core: Time Series Prediction by Adaptive Masking Loss with Representation Consistency [4.047219770183742]
  Time series forecasting plays a pivotal role in critical domains such as energy management and financial markets.<n>This study reveals a counterintuitive phenomenon: appropriately truncating historical data can enhance prediction accuracy.<n>We propose an innovative solution termed Adaptive Masking Loss with Representation Consistency.
  arXiv  Detail & Related papers  (2025-10-22T19:23:53Z)
• Self-Predictive Dynamics for Generalization of Vision-based Reinforcement Learning [14.137070712516005]
  Vision-based reinforcement learning requires efficient and robust representations of image-based observations.<n>We design a Self-Predictive Dynamics (SPD) method to extract task-relevant features efficiently.<n>In a set of MuJoCo visual control tasks and an autonomous driving task (CARLA), SPD outperforms previous studies in complex observations.
  arXiv  Detail & Related papers  (2025-06-05T00:36:54Z)
• Reinforcement Learning under Latent Dynamics: Toward Statistical and Algorithmic Modularity [51.40558987254471]
  Real-world applications of reinforcement learning often involve environments where agents operate on complex, high-dimensional observations. This paper addresses the question of reinforcement learning under $textitgeneral$ latent dynamics from a statistical and algorithmic perspective.
  arXiv  Detail & Related papers  (2024-10-23T14:22:49Z)
• Learning Interpretable Policies in Hindsight-Observable POMDPs through Partially Supervised Reinforcement Learning [57.67629402360924]
  We introduce the Partially Supervised Reinforcement Learning (PSRL) framework. At the heart of PSRL is the fusion of both supervised and unsupervised learning. We show that PSRL offers a potent balance, enhancing model interpretability while preserving, and often significantly outperforming, the performance benchmarks set by traditional methods.
  arXiv  Detail & Related papers  (2024-02-14T16:23:23Z)
• An Empirical Study on the Power of Future Prediction in Partially Observable Environments [15.773444560355694]
  Self-predictive auxiliary tasks have been shown to improve performance in fully observed settings, but their role in partial observability remains underexplored.<n>We introduce $textttDRL2$, an approach that explicitly decouples representation learning from reinforcement learning.<n>Our findings reinforce the idea that future prediction performance serves as a reliable indicator of representation quality and contributes to improved RL performance.
  arXiv  Detail & Related papers  (2024-02-11T04:53:40Z)
• Learning Robust Representations via Bidirectional Transition for Visual Reinforcement Learning [49.23256535551141]
  We introduce a Bidirectional Transition (BiT) model, which leverages the ability to bidirectionally predict environmental transitions both forward and backward to extract reliable representations.<n>Our model demonstrates competitive generalization performance and sample efficiency on two settings of the DeepMind Control suite.
  arXiv  Detail & Related papers  (2023-12-04T14:19:36Z)
• Understanding Self-Predictive Learning for Reinforcement Learning [61.62067048348786]
  We study the learning dynamics of self-predictive learning for reinforcement learning. We propose a novel self-predictive algorithm that learns two representations simultaneously.
  arXiv  Detail & Related papers  (2022-12-06T20:43:37Z)
• Self-Regulated Learning for Egocentric Video Activity Anticipation [147.9783215348252]
  Self-Regulated Learning (SRL) aims to regulate the intermediate representation consecutively to produce representation that emphasizes the novel information in the frame of the current time-stamp. SRL sharply outperforms existing state-of-the-art in most cases on two egocentric video datasets and two third-person video datasets.
  arXiv  Detail & Related papers  (2021-11-23T03:29:18Z)
• Unsupervised Gaze Prediction in Egocentric Videos by Energy-based Surprise Modeling [6.294759639481189]
  Egocentric perception has grown rapidly with the advent of immersive computing devices. Human gaze prediction is an important problem in analyzing egocentric videos. We quantitatively analyze the generalization capabilities of supervised, deep learning models on the egocentric gaze prediction task.
  arXiv  Detail & Related papers  (2020-01-30T21:52:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.