A Survey of In-Context Reinforcement Learning

• URL: http://arxiv.org/abs/2502.07978v1
• Date: Tue, 11 Feb 2025 21:52:19 GMT
• Title: A Survey of In-Context Reinforcement Learning
• Authors: Amir Moeini, Jiuqi Wang, Jacob Beck, Ethan Blaser, Shimon Whiteson, Rohan Chandra, Shangtong Zhang,
• Abstract summary: Some agents can solve new tasks without updating any parameters by simply conditioning on additional context.<n>This paper surveys work on such behavior, known as in-context reinforcement learning.
• Score: 41.74105124619678
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reinforcement learning (RL) agents typically optimize their policies by performing expensive backward passes to update their network parameters. However, some agents can solve new tasks without updating any parameters by simply conditioning on additional context such as their action-observation histories. This paper surveys work on such behavior, known as in-context reinforcement learning.

Related papers

• RAG-RL: Advancing Retrieval-Augmented Generation via RL and Curriculum Learning [11.872929831119661]
  We introduce RAG-RL, the first reasoning language model (RLM) specifically trained for retrieval-augmented generation (RAG) settings. RAG-RL demonstrates that stronger answer generation models can identify relevant contexts within larger sets of retrieved information. We show that curriculum design in the reinforcement learning (RL) post-training process is a powerful approach to enhancing model performance.
  arXiv  Detail & Related papers  (2025-03-17T02:53:42Z)
• From Novice to Expert: LLM Agent Policy Optimization via Step-wise Reinforcement Learning [62.54484062185869]
  We introduce StepAgent, which utilizes step-wise reward to optimize the agent's reinforcement learning process.<n>We propose implicit-reward and inverse reinforcement learning techniques to facilitate agent reflection and policy adjustment.
  arXiv  Detail & Related papers  (2024-11-06T10:35:11Z)
• ReLIC: A Recipe for 64k Steps of In-Context Reinforcement Learning for Embodied AI [44.77897322913095]
  We present ReLIC, a new approach for in-context reinforcement learning for embodied agents. With ReLIC, agents are capable of adapting to new environments using 64,000 steps of in-context experience. We find that ReLIC is capable of few-shot imitation learning despite never being trained with expert demonstrations.
  arXiv  Detail & Related papers  (2024-10-03T17:58:11Z)
• Transformers Can Learn Temporal Difference Methods for In-Context Reinforcement Learning [17.714908233024847]
  reinforcement learning (RL) agents learn to solve new tasks by updating their neural network parameters through interactions with the task environment. Recent works demonstrate that some RL agents, after certain pretraining procedures, can learn to solve unseen new tasks without parameter updates.
  arXiv  Detail & Related papers  (2024-05-22T17:38:16Z)
• Hypernetworks for Zero-shot Transfer in Reinforcement Learning [21.994654567458017]
  Hypernetworks are trained to generate behaviors across a range of unseen task conditions. This work relates to meta RL, contextual RL, and transfer learning. Our method demonstrates significant improvements over baselines from multitask and meta RL approaches.
  arXiv  Detail & Related papers  (2022-11-28T15:48:35Z)
• Jump-Start Reinforcement Learning [68.82380421479675]
  We present a meta algorithm that can use offline data, demonstrations, or a pre-existing policy to initialize an RL policy. In particular, we propose Jump-Start Reinforcement Learning (JSRL), an algorithm that employs two policies to solve tasks. We show via experiments that JSRL is able to significantly outperform existing imitation and reinforcement learning algorithms.
  arXiv  Detail & Related papers  (2022-04-05T17:25:22Z)
• New Insights on Reducing Abrupt Representation Change in Online Continual Learning [69.05515249097208]
  We focus on the change in representations of observed data that arises when previously unobserved classes appear in the incoming data stream. We show that applying Experience Replay causes the newly added classes' representations to overlap significantly with the previous classes. We propose a new method which mitigates this issue by shielding the learned representations from drastic adaptation to accommodate new classes.
  arXiv  Detail & Related papers  (2022-03-08T01:37:00Z)
• What is Going on Inside Recurrent Meta Reinforcement Learning Agents? [63.58053355357644]
  Recurrent meta reinforcement learning (meta-RL) agents are agents that employ a recurrent neural network (RNN) for the purpose of "learning a learning algorithm" We shed light on the internal working mechanisms of these agents by reformulating the meta-RL problem using the Partially Observable Markov Decision Process (POMDP) framework.
  arXiv  Detail & Related papers  (2021-04-29T20:34:39Z)
• Off-Policy Meta-Reinforcement Learning Based on Feature Embedding Spaces [14.029933823101084]
  We propose a novel off-policy meta-RL method, embedding learning and evaluation of uncertainty (ELUE) ELUE learns a belief model over the embedding space and a belief-conditional policy and Q-function. We demonstrate that ELUE outperforms state-of-the-art meta RL methods through experiments on meta-RL benchmarks.
  arXiv  Detail & Related papers  (2021-01-06T05:51:38Z)
• Parrot: Data-Driven Behavioral Priors for Reinforcement Learning [79.32403825036792]
  We propose a method for pre-training behavioral priors that can capture complex input-output relationships observed in successful trials. We show how this learned prior can be used for rapidly learning new tasks without impeding the RL agent's ability to try out novel behaviors.
  arXiv  Detail & Related papers  (2020-11-19T18:47:40Z)
• Keep Doing What Worked: Behavioral Modelling Priors for Offline Reinforcement Learning [25.099754758455415]
  Off-policy reinforcement learning algorithms promise to be applicable in settings where only a fixed data-set of environment interactions is available. Standard off-policy algorithms fail in the batch setting for continuous control.
  arXiv  Detail & Related papers  (2020-02-19T19:21:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.