Training Reinforcement Learning Agents and Humans With Difficulty-Conditioned Generators

• URL: http://arxiv.org/abs/2312.02309v1
• Date: Mon, 4 Dec 2023 19:45:06 GMT
• Title: Training Reinforcement Learning Agents and Humans With Difficulty-Conditioned Generators
• Abstract summary: We adapt Reinforcement Learning (RL) Agents and human learners in parameterized environments by directly modeling difficulty and ability. Inspired by Item Response Theory (IRT), PERM aligns environment difficulty with individual ability, creating a Zone of Proximal Development-based curriculum. We present a two-stage training process that capitalizes on PERM's adaptability, and demonstrate its effectiveness in training RL agents and humans in an empirical study.
• Score: 6.98887755747127
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We adapt Parameterized Environment Response Model (PERM), a method for training both Reinforcement Learning (RL) Agents and human learners in parameterized environments by directly modeling difficulty and ability. Inspired by Item Response Theory (IRT), PERM aligns environment difficulty with individual ability, creating a Zone of Proximal Development-based curriculum. Remarkably, PERM operates without real-time RL updates and allows for offline training, ensuring its adaptability across diverse students. We present a two-stage training process that capitalizes on PERM's adaptability, and demonstrate its effectiveness in training RL agents and humans in an empirical study.

Related papers

• Toward Self-Adaptive Physical AI: Can LLM Agents Manage Long-Horizon Physical Tasks? [1.6822770693792826]
  Large Language Model (LLM) agents offer a promising path toward autonomously managing long-term physical tasks without human intervention.<n>Existing approaches either require substantial data and retraining, or primarily focus on agents operating in the virtual world.<n>We explore the feasibility of building a self-adaptive physical AI agent that manages long-term physical tasks in a zero-shot manner and adapts to environmental changes without human intervention.
  arXiv  Detail & Related papers  (2026-09-11T18:52:40Z)
• Autonomous Continual Learning of Computer-Use Agents for Environment Adaptation [57.65688895630163]
  We introduce ACuRL, an Autonomous Curriculum Reinforcement Learning framework that continually adapts agents to specific environments with zero human data.<n>Our method effectively enables both intra-environment and cross-environment continual learning, yielding 4-22% performance gains without forgetting existing environments.
  arXiv  Detail & Related papers  (2026-02-10T23:06:02Z)
• Scaling Agent Learning via Experience Synthesis [100.42712232390532]
  Reinforcement learning can empower autonomous agents by enabling self-improvement through interaction.<n>But its practical adoption remains challenging due to costly rollouts, limited task diversity, unreliable reward signals, and infrastructure complexity.<n>We introduce DreamGym, the first unified framework designed to synthesize diverse experiences with scalability in mind.
  arXiv  Detail & Related papers  (2025-11-05T18:58:48Z)
• Aligning LLM agents with human learning and adjustment behavior: a dual agent approach [30.79232557548701]
  We introduce a novel dual-agent framework that enables continuous learning and alignment between Large Language Model (LLM) agents and human travelers.<n>Our approach involves a set of LLM traveler agents, equipped with a memory system and a learnable persona, which serve as simulators for human travelers.<n>We show our approach significantly outperforms existing LLM-based methods in both individual behavioral alignment and aggregate simulation accuracy.
  arXiv  Detail & Related papers  (2025-11-02T16:05:33Z)
• Human-in-the-loop Online Rejection Sampling for Robotic Manipulation [55.99788088622936]
  Hi-ORS stabilizes value estimation by filtering out negatively rewarded samples during online fine-tuning.<n>Hi-ORS fine-tunes a pi-base policy to master contact-rich manipulation in just 1.5 hours of real-world training.
  arXiv  Detail & Related papers  (2025-10-30T11:53:08Z)
• Investigating Pedagogical Teacher and Student LLM Agents: Genetic Adaptation Meets Retrieval Augmented Generation Across Learning Style [16.985943868964394]
  Effective teaching requires adapting instructional strategies to accommodate the diverse cognitive and behavioral profiles of students.<n>This paper introduces a novel simulation framework that integrates heterogeneous student agents with a self-optimizing teacher agent.<n>Our results highlight the potential of LLM-driven simulations to inform adaptive teaching practices and provide a testbed for training human educators in data-driven environments.
  arXiv  Detail & Related papers  (2025-05-25T14:45:35Z)
• Ego-Foresight: Agent Visuomotor Prediction as Regularization for RL [34.6883445484835]
  Ego-Foresight is a self-supervised method for disentangling agent and environment based on motion and prediction. We show that visuomotor prediction of the agent provides regularization to the RL algorithm, by encouraging the actions to stay within predictable bounds. We integrate Ego-Foresight with a model-free RL algorithm to solve simulated robotic manipulation tasks, showing an average improvement of 23% in efficiency and 8% in performance.
  arXiv  Detail & Related papers  (2024-05-27T13:32:43Z)
• Large Language Model-based Human-Agent Collaboration for Complex Task Solving [94.3914058341565]
  We introduce the problem of Large Language Models (LLMs)-based human-agent collaboration for complex task-solving. We propose a Reinforcement Learning-based Human-Agent Collaboration method, ReHAC. This approach includes a policy model designed to determine the most opportune stages for human intervention within the task-solving process.
  arXiv  Detail & Related papers  (2024-02-20T11:03:36Z)
• ADAPTER-RL: Adaptation of Any Agent using Reinforcement Learning [0.0]
  adapters have proven effective in supervised learning contexts such as natural language processing and computer vision. This paper presents an innovative adaptation strategy that demonstrates enhanced training efficiency and improvement of the base-agent. Our proposed universal approach is not only compatible with pre-trained neural networks but also with rule-based agents, offering a means to integrate human expertise.
  arXiv  Detail & Related papers  (2023-11-20T04:54:51Z)
• Grow Your Limits: Continuous Improvement with Real-World RL for Robotic Locomotion [66.69666636971922]
  We present APRL, a policy regularization framework that modulates the robot's exploration over the course of training. APRL enables a quadrupedal robot to efficiently learn to walk entirely in the real world within minutes.
  arXiv  Detail & Related papers  (2023-10-26T17:51:46Z)
• Improving Generalization in Reinforcement Learning Training Regimes for Social Robot Navigation [5.475804640008192]
  We propose a method to improve the generalization performance of RL social navigation methods using curriculum learning. Our results show that the use of curriculum learning in training can be used to achieve better generalization performance than previous training methods.
  arXiv  Detail & Related papers  (2023-08-29T00:00:18Z)
• Transferable Curricula through Difficulty Conditioned Generators [8.140037969280716]
  Reinforcement learning (RL) has demonstrated superhuman performance in complex tasks such as Starcraft, Go, Chess etc. Recent methods in curricula generation focuses on training RL agents efficiently, yet such methods rely on surrogate measures to track student progress. We introduce a method named. Environment Response Model (PERM) that shows promising results in training RL agents in parameterized environments.
  arXiv  Detail & Related papers  (2023-06-22T16:45:45Z)
• Training and Evaluation of Deep Policies using Reinforcement Learning and Generative Models [67.78935378952146]
  GenRL is a framework for solving sequential decision-making problems. It exploits the combination of reinforcement learning and latent variable generative models. We experimentally determine the characteristics of generative models that have most influence on the performance of the final policy training.
  arXiv  Detail & Related papers  (2022-04-18T22:02:32Z)
• Cloud-Edge Training Architecture for Sim-to-Real Deep Reinforcement Learning [0.8399688944263843]
  Deep reinforcement learning (DRL) is a promising approach to solve complex control tasks by learning policies through interactions with the environment. Sim-to-real approaches leverage simulations to pretrain DRL policies and then deploy them in the real world. This work proposes a distributed cloud-edge architecture to train DRL agents in the real world in real-time.
  arXiv  Detail & Related papers  (2022-03-04T10:27:01Z)
• Autonomous Reinforcement Learning: Formalism and Benchmarking [106.25788536376007]
  Real-world embodied learning, such as that performed by humans and animals, is situated in a continual, non-episodic world. Common benchmark tasks in RL are episodic, with the environment resetting between trials to provide the agent with multiple attempts. This discrepancy presents a major challenge when attempting to take RL algorithms developed for episodic simulated environments and run them on real-world platforms.
  arXiv  Detail & Related papers  (2021-12-17T16:28:06Z)
• Multitask Adaptation by Retrospective Exploration with Learned World Models [77.34726150561087]
  We propose a meta-learned addressing model called RAMa that provides training samples for the MBRL agent taken from task-agnostic storage. The model is trained to maximize the expected agent's performance by selecting promising trajectories solving prior tasks from the storage.
  arXiv  Detail & Related papers  (2021-10-25T20:02:57Z)
• Robust Reinforcement Learning via Adversarial training with Langevin Dynamics [51.234482917047835]
  We introduce a sampling perspective to tackle the challenging task of training robust Reinforcement Learning (RL) agents. We present a novel, scalable two-player RL algorithm, which is a sampling variant of the two-player policy method.
  arXiv  Detail & Related papers  (2020-02-14T14:59:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.