Toddler-Guidance Learning: Impacts of Critical Period on Multimodal AI
Agents
- URL: http://arxiv.org/abs/2201.04990v1
- Date: Wed, 12 Jan 2022 10:57:40 GMT
- Title: Toddler-Guidance Learning: Impacts of Critical Period on Multimodal AI
Agents
- Authors: Junseok Park, Kwanyoung Park, Hyunseok Oh, Ganghun Lee, Minsu Lee,
Youngki Lee, Byoung-Tak Zhang
- Abstract summary: We adapt the notion of critical periods to learning in AI agents and investigate the critical period in the virtual environment for AI agents.
We build up a toddler-like environment with VECA toolkit to mimic human toddlers' learning characteristics.
We evaluate the impact of critical periods on AI agents from two perspectives: how and when they are guided best in both uni- and multimodal learning.
- Score: 18.610737380842494
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Critical periods are phases during which a toddler's brain develops in
spurts. To promote children's cognitive development, proper guidance is
critical in this stage. However, it is not clear whether such a critical period
also exists for the training of AI agents. Similar to human toddlers,
well-timed guidance and multimodal interactions might significantly enhance the
training efficiency of AI agents as well. To validate this hypothesis, we adapt
this notion of critical periods to learning in AI agents and investigate the
critical period in the virtual environment for AI agents. We formalize the
critical period and Toddler-guidance learning in the reinforcement learning
(RL) framework. Then, we built up a toddler-like environment with VECA toolkit
to mimic human toddlers' learning characteristics. We study three discrete
levels of mutual interaction: weak-mentor guidance (sparse reward), moderate
mentor guidance (helper-reward), and mentor demonstration (behavioral cloning).
We also introduce the EAVE dataset consisting of 30,000 real-world images to
fully reflect the toddler's viewpoint. We evaluate the impact of critical
periods on AI agents from two perspectives: how and when they are guided best
in both uni- and multimodal learning. Our experimental results show that both
uni- and multimodal agents with moderate mentor guidance and critical period on
1 million and 2 million training steps show a noticeable improvement. We
validate these results with transfer learning on the EAVE dataset and find the
performance advancement on the same critical period and the guidance.
Related papers
- Interactive Autonomous Navigation with Internal State Inference and
Interactivity Estimation [58.21683603243387]
We propose three auxiliary tasks with relational-temporal reasoning and integrate them into the standard Deep Learning framework.
These auxiliary tasks provide additional supervision signals to infer the behavior patterns other interactive agents.
Our approach achieves robust and state-of-the-art performance in terms of standard evaluation metrics.
arXiv Detail & Related papers (2023-11-27T18:57:42Z) - Leveraging Deep Reinforcement Learning for Metacognitive Interventions
across Intelligent Tutoring Systems [7.253181280137071]
This work compares two approaches to provide metacognitive interventions across Intelligent Tutoring Systems (ITSs)
In two consecutive semesters, we conducted two classroom experiments: Exp. 1 used a classic artificial intelligence approach to classify students into different metacognitive groups and provide static interventions based on their classified groups.
In Exp. 2, we leveraged Deep Reinforcement Learning (DRL) to provide adaptive interventions that consider the dynamic changes in the student's metacognitive levels.
arXiv Detail & Related papers (2023-04-17T12:10:50Z) - On the Importance of Critical Period in Multi-stage Reinforcement
Learning [18.610737380842494]
In recent studies, an AI agent exhibited a learning period similar to human's critical period.
We propose multi-stage reinforcement learning to emphasize finding appropriate stimulus.
arXiv Detail & Related papers (2022-08-09T15:17:22Z) - Human Decision Makings on Curriculum Reinforcement Learning with
Difficulty Adjustment [52.07473934146584]
We guide the curriculum reinforcement learning results towards a preferred performance level that is neither too hard nor too easy via learning from the human decision process.
Our system is highly parallelizable, making it possible for a human to train large-scale reinforcement learning applications.
It shows reinforcement learning performance can successfully adjust in sync with the human desired difficulty level.
arXiv Detail & Related papers (2022-08-04T23:53:51Z) - Autonomous Reinforcement Learning: Formalism and Benchmarking [106.25788536376007]
Real-world embodied learning, such as that performed by humans and animals, is situated in a continual, non-episodic world.
Common benchmark tasks in RL are episodic, with the environment resetting between trials to provide the agent with multiple attempts.
This discrepancy presents a major challenge when attempting to take RL algorithms developed for episodic simulated environments and run them on real-world platforms.
arXiv Detail & Related papers (2021-12-17T16:28:06Z) - Persistent Reinforcement Learning via Subgoal Curricula [114.83989499740193]
Value-accelerated Persistent Reinforcement Learning (VaPRL) generates a curriculum of initial states.
VaPRL reduces the interventions required by three orders of magnitude compared to episodic reinforcement learning.
arXiv Detail & Related papers (2021-07-27T16:39:45Z) - PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via
Relabeling Experience and Unsupervised Pre-training [94.87393610927812]
We present an off-policy, interactive reinforcement learning algorithm that capitalizes on the strengths of both feedback and off-policy learning.
We demonstrate that our approach is capable of learning tasks of higher complexity than previously considered by human-in-the-loop methods.
arXiv Detail & Related papers (2021-06-09T14:10:50Z) - Towards Teachable Autotelic Agents [21.743801780657435]
Teachable autotelic agents (TAA) are agents that learn from both internal and teaching signals.
This paper presents a roadmap towards the design of teachable autonomous agents.
arXiv Detail & Related papers (2021-05-25T14:28:58Z) - Bridging the Imitation Gap by Adaptive Insubordination [88.35564081175642]
We show that when the teaching agent makes decisions with access to privileged information, this information is marginalized during imitation learning.
We propose 'Adaptive Insubordination' (ADVISOR) to address this gap.
ADVISOR dynamically weights imitation and reward-based reinforcement learning losses during training, enabling on-the-fly switching between imitation and exploration.
arXiv Detail & Related papers (2020-07-23T17:59:57Z) - Human AI interaction loop training: New approach for interactive
reinforcement learning [0.0]
Reinforcement Learning (RL) in various decision-making tasks of machine learning provides effective results with an agent learning from a stand-alone reward function.
RL presents unique challenges with large amounts of environment states and action spaces, as well as in the determination of rewards.
Imitation Learning (IL) offers a promising solution for those challenges using a teacher.
arXiv Detail & Related papers (2020-03-09T15:27:48Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.