An Introduction to Deep Reinforcement and Imitation Learning

• URL: http://arxiv.org/abs/2512.08052v2
• Date: Thu, 11 Dec 2025 00:05:29 GMT
• Title: An Introduction to Deep Reinforcement and Imitation Learning
• Authors: Pedro Santana,
• Abstract summary: This document introduces DRL and DIL in the context of embodied agents.<n>It is self-contained, presenting all necessary mathematical and machine learning concepts as they are needed.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Embodied agents, such as robots and virtual characters, must continuously select actions to execute tasks effectively, solving complex sequential decision-making problems. Given the difficulty of designing such controllers manually, learning-based approaches have emerged as promising alternatives, most notably Deep Reinforcement Learning (DRL) and Deep Imitation Learning (DIL). DRL leverages reward signals to optimize behavior, while DIL uses expert demonstrations to guide learning. This document introduces DRL and DIL in the context of embodied agents, adopting a concise, depth-first approach to the literature. It is self-contained, presenting all necessary mathematical and machine learning concepts as they are needed. It is not intended as a survey of the field; rather, it focuses on a small set of foundational algorithms and techniques, prioritizing in-depth understanding over broad coverage. The material ranges from Markov Decision Processes to REINFORCE and Proximal Policy Optimization (PPO) for DRL, and from Behavioral Cloning to Dataset Aggregation (DAgger) and Generative Adversarial Imitation Learning (GAIL) for DIL.

Related papers

• Search-R3: Unifying Reasoning and Embedding Generation in Large Language Models [11.39711340224126]
  Search-R3 is a novel framework that adapts Large Language Models to generate search embeddings as a direct output of their reasoning process.<n>Our approach exploits LLMs' chain-of-thought capabilities, allowing them to produce more effective embeddings by reasoning step-by-step through complex semantic analyses.
  arXiv  Detail & Related papers  (2025-10-08T14:16:20Z)
• RLAD: Training LLMs to Discover Abstractions for Solving Reasoning Problems [98.98963933669751]
  We train models to be capable of proposing multiple abstractions given a problem, followed by RL that incentivizes building a solution.<n>This results in a two-player RL training paradigm, abbreviated as RLAD, that jointly trains an abstraction generator and a solution generator.<n>We show that allocating more test-time compute to generating abstractions is more beneficial for performance than generating more solutions at large test budgets.
  arXiv  Detail & Related papers  (2025-10-02T17:44:23Z)
• Does Machine Unlearning Truly Remove Knowledge? [80.83986295685128]
  We introduce a comprehensive auditing framework for unlearning evaluation comprising three benchmark datasets, six unlearning algorithms, and five prompt-based auditing methods.<n>We evaluate the effectiveness and robustness of different unlearning strategies.
  arXiv  Detail & Related papers  (2025-05-29T09:19:07Z)
• A Practical Introduction to Deep Reinforcement Learning [4.701599716969864]
  This tutorial aims to provide a concise, intuitive, and practical introduction to deep reinforcement learning (DRL)<n>We organize all algorithms under the Generalized Policy Iteration (GPI) framework, offering readers a unified and systematic perspective.<n>Instead of lengthy theoretical proofs, we emphasize intuitive explanations, illustrative examples, and practical engineering techniques.
  arXiv  Detail & Related papers  (2025-05-13T07:19:16Z)
• A Comprehensive Survey of Reinforcement Learning: From Algorithms to Practical Challenges [2.2448567386846916]
  Reinforcement Learning (RL) has emerged as a powerful paradigm in Artificial Intelligence (AI)<n>This paper presents a comprehensive survey of RL, meticulously analyzing a wide range of algorithms.<n>We offer practical insights into the selection and implementation of RL algorithms, addressing common challenges like convergence, stability, and the exploration-exploitation dilemma.
  arXiv  Detail & Related papers  (2024-11-28T03:53:14Z)
• Introduction to Reinforcement Learning [2.52299400625445]
  Reinforcement Learning (RL) focuses on training agents to make decisions by interacting with their environment to maximize cumulative rewards.<n>This paper provides an overview of RL, covering its core concepts, methodologies, and resources for further learning.
  arXiv  Detail & Related papers  (2024-08-13T23:08:06Z)
• How Can LLM Guide RL? A Value-Based Approach [68.55316627400683]
  Reinforcement learning (RL) has become the de facto standard practice for sequential decision-making problems by improving future acting policies with feedback. Recent developments in large language models (LLMs) have showcased impressive capabilities in language understanding and generation, yet they fall short in exploration and self-improvement capabilities. We develop an algorithm named LINVIT that incorporates LLM guidance as a regularization factor in value-based RL, leading to significant reductions in the amount of data needed for learning.
  arXiv  Detail & Related papers  (2024-02-25T20:07:13Z)
• Jump-Start Reinforcement Learning [68.82380421479675]
  We present a meta algorithm that can use offline data, demonstrations, or a pre-existing policy to initialize an RL policy. In particular, we propose Jump-Start Reinforcement Learning (JSRL), an algorithm that employs two policies to solve tasks. We show via experiments that JSRL is able to significantly outperform existing imitation and reinforcement learning algorithms.
  arXiv  Detail & Related papers  (2022-04-05T17:25:22Z)
• RvS: What is Essential for Offline RL via Supervised Learning? [77.91045677562802]
  Recent work has shown that supervised learning alone, without temporal difference (TD) learning, can be remarkably effective for offline RL. In every environment suite we consider simply maximizing likelihood with two-layer feedforward is competitive. They also probe the limits of existing RvS methods, which are comparatively weak on random data.
  arXiv  Detail & Related papers  (2021-12-20T18:55:16Z)
• Learning Dexterous Manipulation from Suboptimal Experts [69.8017067648129]
  Relative Entropy Q-Learning (REQ) is a simple policy algorithm that combines ideas from successful offline and conventional RL algorithms. We show how REQ is also effective for general off-policy RL, offline RL, and RL from demonstrations.
  arXiv  Detail & Related papers  (2020-10-16T18:48:49Z)
• Self-Paced Deep Reinforcement Learning [42.467323141301826]
  Curriculum reinforcement learning (CRL) improves the learning speed and stability of an agent by exposing it to a tailored series of tasks throughout learning. Despite empirical successes, an open question in CRL is how to automatically generate a curriculum for a given reinforcement learning (RL) agent, avoiding manual design. We propose an answer by interpreting the curriculum generation as an inference problem, where distributions over tasks are progressively learned to approach the target task. This approach leads to an automatic curriculum generation, whose pace is controlled by the agent, with solid theoretical motivation and easily integrated with deep RL algorithms.
  arXiv  Detail & Related papers  (2020-04-24T15:48:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.