Intrinsic Vicarious Conditioning for Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2605.12224v1
• Date: Tue, 12 May 2026 15:01:37 GMT
• Title: Intrinsic Vicarious Conditioning for Deep Reinforcement Learning
• Abstract summary: Direct conditioning limits agents' capacity by restricting how they learn from the environment as well as from others.<n>Off-policy or learn-by-example methods can learn from demonstrators' representations, but they require access to the demonstrating agent's policies or their reward functions.<n>Our work overcomes this direct sampling limitation by introducing vicarious conditioning as an intrinsic reward mechanism.
• Score: 1.5466093929120726
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Advancements in reinforcement learning have produced a variety of complex and useful intrinsic driving forces; crucially, these drivers operate under a direct conditioning paradigm. This form of conditioning limits our agents' capacity by restricting how they learn from the environment as well as from others. Off-policy or learn-by-example methods can learn from demonstrators' representations, but they require access to the demonstrating agent's policies or their reward functions. Our work overcomes this direct sampling limitation by introducing vicarious conditioning as an intrinsic reward mechanism. We draw from psychological and biological literature to provide a foundation for vicarious conditioning and use memory-based methods to implement its four steps: attention, retention, reproduction, and reinforcement. Crucially, our vicarious conditioning paradigms support low-shot learning and do not require the demonstrator agent's policy nor its reward functions. We evaluate our approach in the MiniWorld Sidewalk environment, one of the few public environments that features a non-descriptive terminal condition (no reward provided upon agent death), and extend it to Box2D's CarRacing environment. Our results across both environments demonstrate that vicarious conditioning enables longer episode lengths by discouraging the agent from non-descriptive terminal conditions and guiding the agent toward desirable states. Overall, this work emulates a cognitively-plausible learning paradigm better suited to problems such as single-life learning or continual learning.

Related papers

• Revealing Behavioral Plasticity in Large Language Models: A Token-Conditional Perspective [63.9266662672413]
  We show that Large Language Models (LLMs) possess intrinsic behavioral plasticity that can be exposed through token-conditional generation and stabilized via reinforcement learning.<n>We propose Token-Conditioned Reinforcement Learning (ToCoRL) to transform transient inference-time adaptations into stable and learnable behavioral patterns.
  arXiv  Detail & Related papers  (2026-03-09T13:56:53Z)
• Agent Learning via Early Experience [93.83579011718858]
  A long-term goal of language agents is to learn and improve through their own experience, ultimately outperforming humans in complex, real-world tasks.<n>Most current agents rely on supervised fine-tuning on expert data, which is challenging to scale and generalizes poorly.<n>We study two strategies of using such data: (1) Implicit world modeling, which uses collected states to ground the policy in environment dynamics; and (2) Self-reflection, where the agent learns from its suboptimal actions to improve reasoning and decision-making.
  arXiv  Detail & Related papers  (2025-10-09T17:59:17Z)
• Avoiding Death through Fear Intrinsic Conditioning [48.07595141865156]
  We introduce an intrinsic reward function inspired by early amygdala development and produce this intrinsic reward through a novel memory-augmented neural network architecture.<n>We show how this intrinsic motivation serves to deter exploration of terminal states and results in avoidance behavior similar to fear conditioning observed in animals.
  arXiv  Detail & Related papers  (2025-06-05T19:24:51Z)
• Goal-Conditioned Reinforcement Learning in the Presence of an Adversary [0.0]
  Reinforcement learning has seen increasing applications in real-world contexts over the past few years. A common approach to combat this is to train agents in the presence of an adversary. An adversary acts to destabilise the agent, which learns a more robust policy and can better handle realistic conditions. We present DigitFlip and CLEVR-Play, two novel goal-conditioned environments that support acting against an adversary.
  arXiv  Detail & Related papers  (2022-11-13T15:40:01Z)
• Safer Autonomous Driving in a Stochastic, Partially-Observable Environment by Hierarchical Contingency Planning [10.971411555103574]
  An intelligent agent should be prepared to anticipate a change in its belief of the environment state. This is especially the case for autonomous vehicles (AVs) navigating real-world situations where safety is paramount. We show that our approach results in robust, safe behaviour in a partially observable, safe environment, generalizing well over environment not seen during training.
  arXiv  Detail & Related papers  (2022-04-13T16:47:00Z)
• Information is Power: Intrinsic Control via Information Capture [110.3143711650806]
  We argue that a compact and general learning objective is to minimize the entropy of the agent's state visitation estimated using a latent state-space model. This objective induces an agent to both gather information about its environment, corresponding to reducing uncertainty, and to gain control over its environment, corresponding to reducing the unpredictability of future world states.
  arXiv  Detail & Related papers  (2021-12-07T18:50:42Z)
• Online reinforcement learning with sparse rewards through an active inference capsule [62.997667081978825]
  This paper introduces an active inference agent which minimizes the novel free energy of the expected future. Our model is capable of solving sparse-reward problems with a very high sample efficiency. We also introduce a novel method for approximating the prior model from the reward function, which simplifies the expression of complex objectives.
  arXiv  Detail & Related papers  (2021-06-04T10:03:36Z)
• Ecological Reinforcement Learning [76.9893572776141]
  We study the kinds of environment properties that can make learning under such conditions easier. understanding how properties of the environment impact the performance of reinforcement learning agents can help us to structure our tasks in ways that make learning tractable.
  arXiv  Detail & Related papers  (2020-06-22T17:55:03Z)
• RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environments [15.736899098702972]
  We propose a novel type of intrinsic reward which encourages the agent to take actions that lead to significant changes in its learned state representation. We evaluate our method on multiple challenging procedurally-generated tasks in MiniGrid.
  arXiv  Detail & Related papers  (2020-02-27T18:03:16Z)
• oIRL: Robust Adversarial Inverse Reinforcement Learning with Temporally Extended Actions [37.66289166905027]
  Explicit engineering of reward functions for given environments has been a major hindrance to reinforcement learning methods. We propose an algorithm that learns hierarchical disentangled rewards with a policy over options.
  arXiv  Detail & Related papers  (2020-02-20T22:21:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.