DreamerV3-XP: Optimizing exploration through uncertainty estimation

• URL: http://arxiv.org/abs/2510.21418v1
• Date: Fri, 24 Oct 2025 12:58:27 GMT
• Title: DreamerV3-XP: Optimizing exploration through uncertainty estimation
• Abstract summary: We introduce DreamerV3-XP, an extension of DreamerV3 that improves exploration and learning efficiency.<n>DreamerV3-XP is evaluated on a subset of Atari100k and DeepMind Control Visual Benchmark tasks.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce DreamerV3-XP, an extension of DreamerV3 that improves exploration and learning efficiency. This includes (i) a prioritized replay buffer, scoring trajectories by return, reconstruction loss, and value error and (ii) an intrinsic reward based on disagreement over predicted environment rewards from an ensemble of world models. DreamerV3-XP is evaluated on a subset of Atari100k and DeepMind Control Visual Benchmark tasks, confirming the original DreamerV3 results and showing that our extensions lead to faster learning and lower dynamics model loss, particularly in sparse-reward settings.

Related papers

• Improving Vision-language Models with Perception-centric Process Reward Models [94.51962346430248]
  We propose Perceval, a process reward model (PRM) that enables token-level error grounding.<n>Perceval is trained with perception-intensive supervised training data.<n>We then integrate Perceval into the RL training process to train the policy models.
  arXiv  Detail & Related papers  (2026-04-27T15:08:02Z)
• CDE: Curiosity-Driven Exploration for Efficient Reinforcement Learning in Large Language Models [85.315711639214]
  We introduce Curiosity-Driven Exploration (CDE), a framework that leverages the model's own intrinsic sense of curiosity to guide exploration.<n>For the actor, we use perplexity over its generated response, and for the critic, we use the variance of value estimates from a multi-head architecture.<n>Our theoretical analysis shows that the actor-wise bonus inherently penalizes overconfident errors and promotes diversity among correct responses.
  arXiv  Detail & Related papers  (2025-09-11T17:59:17Z)
• TransDreamerV3: Implanting Transformer In DreamerV3 [0.0]
  This paper introduces TransDreamerV3, a reinforcement learning model that enhances the DreamerV3 architecture by integrating a transformer encoder.<n>We conducted experiments on Atari-Boxing, Atari-Freeway, Atari-Pong, and Crafter tasks, where TransDreamerV3 demonstrated improved performance over DreamerV3.
  arXiv  Detail & Related papers  (2025-06-20T16:09:17Z)
• One RL to See Them All: Visual Triple Unified Reinforcement Learning [92.90120580989839]
  We propose V-Triune, a Visual Triple Unified Reinforcement Learning system that enables visual reasoning and perception tasks within a single training pipeline.<n>V-Triune comprises triple complementary components: Sample-Level Datashelf (to unify diverse task inputs), Verifier-Level Reward (to deliver custom rewards via specialized verifiers).<n>We introduce a novel Dynamic IoU reward, which provides adaptive, progressive, and definite feedback for perception tasks handled by V-Triune.
  arXiv  Detail & Related papers  (2025-05-23T17:41:14Z)
• World Model Agents with Change-Based Intrinsic Motivation [0.0]
  Intrinsic motivation and transfer learning have emerged as promising strategies to address this issue.<n>Change Based Exploration Transfer (CBET) has shown potential in addressing sparse feedback but its effectiveness with modern algorithms remains understudied.
  arXiv  Detail & Related papers  (2025-03-26T23:40:03Z)
• Robust3D-CIL: Robust Class-Incremental Learning for 3D Perception [7.507868991415516]
  3D perception plays a crucial role in real-world applications such as autonomous driving, robotics, and AR/VR.<n>Adopting class-incremental learning (CIL) becomes particularly essential.<n>We introduce a point cloud downsampling-based replay method to utilize the limited replay buffer memory more efficiently.<n>Our method improves the performance of replay-based CIL baselines by 2% to 11%, proving its effectiveness and promising potential for real-world 3D applications.
  arXiv  Detail & Related papers  (2025-03-18T03:36:17Z)
• Benchmarking and Improving Bird's Eye View Perception Robustness in Autonomous Driving [55.93813178692077]
  We present RoboBEV, an extensive benchmark suite designed to evaluate the resilience of BEV algorithms.<n>We assess 33 state-of-the-art BEV-based perception models spanning tasks like detection, map segmentation, depth estimation, and occupancy prediction.<n>Our experimental results also underline the efficacy of strategies like pre-training and depth-free BEV transformations in enhancing robustness against out-of-distribution data.
  arXiv  Detail & Related papers  (2024-05-27T17:59:39Z)
• MuDreamer: Learning Predictive World Models without Reconstruction [58.0159270859475]
  We present MuDreamer, a robust reinforcement learning agent that builds upon the DreamerV3 algorithm by learning a predictive world model without the need for reconstructing input signals. Our method achieves comparable performance on the Atari100k benchmark while benefiting from faster training.
  arXiv  Detail & Related papers  (2024-05-23T22:09:01Z)
• NeRF-Det++: Incorporating Semantic Cues and Perspective-aware Depth Supervision for Indoor Multi-View 3D Detection [72.0098999512727]
  NeRF-Det has achieved impressive performance in indoor multi-view 3D detection by utilizing NeRF to enhance representation learning. We present three corresponding solutions, including semantic enhancement, perspective-aware sampling, and ordinal depth supervision. The resulting algorithm, NeRF-Det++, has exhibited appealing performance in the ScanNetV2 and AR KITScenes datasets.
  arXiv  Detail & Related papers  (2024-02-22T11:48:06Z)
• Rewarding Episodic Visitation Discrepancy for Exploration in Reinforcement Learning [64.8463574294237]
  We propose Rewarding Episodic Visitation Discrepancy (REVD) as an efficient and quantified exploration method. REVD provides intrinsic rewards by evaluating the R'enyi divergence-based visitation discrepancy between episodes. It is tested on PyBullet Robotics Environments and Atari games.
  arXiv  Detail & Related papers  (2022-09-19T08:42:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.