Improving Experience Replay through Modeling of Similar Transitions' Sets

• URL: http://arxiv.org/abs/2111.06907v1
• Date: Fri, 12 Nov 2021 19:27:15 GMT
• Title: Improving Experience Replay through Modeling of Similar Transitions' Sets
• Authors: Daniel Eug\^enio Neves, Jo\~ao Pedro Oliveira Batisteli, Eduardo Felipe Lopes, Lucila Ishitani and Zenilton Kleber Gon\c{c}alves do Patroc\'inio J\'unior (Pontif\'icia Universidade Cat\'olica de Minas Gerais, Belo Horizonte, Brazil)
• Abstract summary: We propose and evaluate a new reinforcement learning method, COMPact Experience Replay (COMPER) Our objective is to reduce the required number of experiences to agent training regarding the total accumulated rewarding in the long run. We report detailed results from five training trials of COMPER for just 100,000 frames and about 25,000 iterations with a small experiences memory.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this work, we propose and evaluate a new reinforcement learning method, COMPact Experience Replay (COMPER), which uses temporal difference learning with predicted target values based on recurrence over sets of similar transitions, and a new approach for experience replay based on two transitions memories. Our objective is to reduce the required number of experiences to agent training regarding the total accumulated rewarding in the long run. Its relevance to reinforcement learning is related to the small number of observations that it needs to achieve results similar to that obtained by relevant methods in the literature, that generally demand millions of video frames to train an agent on the Atari 2600 games. We report detailed results from five training trials of COMPER for just 100,000 frames and about 25,000 iterations with a small experiences memory on eight challenging games of Arcade Learning Environment (ALE). We also present results for a DQN agent with the same experimental protocol on the same games set as the baseline. To verify the performance of COMPER on approximating a good policy from a smaller number of observations, we also compare its results with that obtained from millions of frames presented on the benchmark of ALE.

Related papers

• REBAR: Retrieval-Based Reconstruction for Time-series Contrastive Learning [64.08293076551601]
  We propose a novel method of using a learned measure for identifying positive pairs. Our Retrieval-Based Reconstruction measure measures the similarity between two sequences. We show that the REBAR error is a predictor of mutual class membership.
  arXiv  Detail & Related papers  (2023-11-01T13:44:45Z)
• Pixel to policy: DQN Encoders for within & cross-game reinforcement learning [0.0]
  Reinforcement Learning can be applied to various tasks, and environments. Many environments have a similar structure, which can be exploited to improve RL performance on other tasks. This work explores as well as compares the performance between RL models being trained from the scratch and on different approaches of transfer learning.
  arXiv  Detail & Related papers  (2023-08-01T06:29:33Z)
• COLO: A Contrastive Learning based Re-ranking Framework for One-Stage Summarization [84.70895015194188]
  We propose a Contrastive Learning based re-ranking framework for one-stage summarization called COLO. COLO boosts the extractive and abstractive results of one-stage systems on CNN/DailyMail benchmark to 44.58 and 46.33 ROUGE-1 score.
  arXiv  Detail & Related papers  (2022-09-29T06:11:21Z)
• Convergence Results For Q-Learning With Experience Replay [51.11953997546418]
  We provide a convergence rate guarantee, and discuss how it compares to the convergence of Q-learning depending on important parameters such as the frequency and number of iterations of replay. We also provide theoretical evidence showing when we might expect this to strictly improve performance, by introducing and analyzing a simple class of MDPs.
  arXiv  Detail & Related papers  (2021-12-08T10:22:49Z)
• Improving Experience Replay with Successor Representation [0.0]
  Prioritized experience replay is a reinforcement learning technique shown to speed up learning. Recent work in neuroscience suggests that, in biological organisms, replay is prioritized by both gain and need.
  arXiv  Detail & Related papers  (2021-11-29T05:25:54Z)
• CoCon: Cooperative-Contrastive Learning [52.342936645996765]
  Self-supervised visual representation learning is key for efficient video analysis. Recent success in learning image representations suggests contrastive learning is a promising framework to tackle this challenge. We introduce a cooperative variant of contrastive learning to utilize complementary information across views.
  arXiv  Detail & Related papers  (2021-04-30T05:46:02Z)
• Learning to Sample with Local and Global Contexts in Experience Replay Buffer [135.94190624087355]
  We propose a new learning-based sampling method that can compute the relative importance of transition. We show that our framework can significantly improve the performance of various off-policy reinforcement learning methods.
  arXiv  Detail & Related papers  (2020-07-14T21:12:56Z)
• Memory-Augmented Relation Network for Few-Shot Learning [114.47866281436829]
  In this work, we investigate a new metric-learning method, Memory-Augmented Relation Network (MRN) In MRN, we choose the samples that are visually similar from the working context, and perform weighted information propagation to attentively aggregate helpful information from chosen ones to enhance its representation. We empirically demonstrate that MRN yields significant improvement over its ancestor and achieves competitive or even better performance when compared with other few-shot learning approaches.
  arXiv  Detail & Related papers  (2020-05-09T10:09:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.