Foundations for Transfer in Reinforcement Learning: A Taxonomy of
Knowledge Modalities
- URL: http://arxiv.org/abs/2312.01939v1
- Date: Mon, 4 Dec 2023 14:55:58 GMT
- Title: Foundations for Transfer in Reinforcement Learning: A Taxonomy of
Knowledge Modalities
- Authors: Markus Wulfmeier, Arunkumar Byravan, Sarah Bechtle, Karol Hausman,
Nicolas Heess
- Abstract summary: We look at opportunities and challenges in refining the generalisation and transfer of knowledge.
Within the domain of reinforcement learning (RL), the representation of knowledge manifests through various modalities.
This taxonomy systematically targets these modalities and frames its discussion based on their inherent properties and alignment with different objectives and mechanisms for transfer.
- Score: 28.65224261733876
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Contemporary artificial intelligence systems exhibit rapidly growing
abilities accompanied by the growth of required resources, expansive datasets
and corresponding investments into computing infrastructure. Although earlier
successes predominantly focus on constrained settings, recent strides in
fundamental research and applications aspire to create increasingly general
systems. This evolving landscape presents a dual panorama of opportunities and
challenges in refining the generalisation and transfer of knowledge - the
extraction from existing sources and adaptation as a comprehensive foundation
for tackling new problems. Within the domain of reinforcement learning (RL),
the representation of knowledge manifests through various modalities, including
dynamics and reward models, value functions, policies, and the original data.
This taxonomy systematically targets these modalities and frames its discussion
based on their inherent properties and alignment with different objectives and
mechanisms for transfer. Where possible, we aim to provide coarse guidance
delineating approaches which address requirements such as limiting environment
interactions, maximising computational efficiency, and enhancing generalisation
across varying axes of change. Finally, we analyse reasons contributing to the
prevalence or scarcity of specific forms of transfer, the inherent potential
behind pushing these frontiers, and underscore the significance of
transitioning from designed to learned transfer.
Related papers
- Investigating the Role of Instruction Variety and Task Difficulty in Robotic Manipulation Tasks [50.75902473813379]
This work introduces a comprehensive evaluation framework that systematically examines the role of instructions and inputs in the generalisation abilities of such models.
The proposed framework uncovers the resilience of multimodal models to extreme instruction perturbations and their vulnerability to observational changes.
arXiv Detail & Related papers (2024-07-04T14:36:49Z) - On the Role of Entity and Event Level Conceptualization in Generalizable Reasoning: A Survey of Tasks, Methods, Applications, and Future Directions [46.63556358247516]
Entity- and event-level conceptualization plays a pivotal role in generalizable reasoning.
There is currently a lack of a systematic overview that comprehensively examines existing works in the definition, execution, and application of conceptualization.
We present the first comprehensive survey of 150+ papers, categorizing various definitions, resources, methods, and downstream applications related to conceptualization into a unified taxonomy.
arXiv Detail & Related papers (2024-06-16T10:32:41Z) - Position: Foundation Agents as the Paradigm Shift for Decision Making [24.555816843983003]
We advocate for the construction of foundation agents as a transformative shift in the learning paradigm of agents.
We specify the roadmap of foundation agents from large interactive data collection or generation to self-supervised pretraining and adaptation.
arXiv Detail & Related papers (2024-05-27T09:54:50Z) - Domain Generalization through Meta-Learning: A Survey [6.524870790082051]
Deep neural networks (DNNs) have revolutionized artificial intelligence but often lack performance when faced with out-of-distribution (OOD) data.
This survey paper delves into the realm of meta-learning with a focus on its contribution to domain generalization.
arXiv Detail & Related papers (2024-04-03T14:55:17Z) - Investigating Continual Pretraining in Large Language Models: Insights
and Implications [9.591223887442704]
This paper studies the evolving domain of Continual Learning in large language models (LLMs)
Our primary emphasis is on continual domain-adaptive pretraining, a process designed to equip LLMs with the ability to integrate new information from various domains.
We examine the impact of model size on learning efficacy and forgetting, as well as how the progression and similarity of emerging domains affect the knowledge transfer within these models.
arXiv Detail & Related papers (2024-02-27T10:47:24Z) - A Recent Survey of Heterogeneous Transfer Learning [15.830786437956144]
heterogeneous transfer learning has become a vital strategy in various tasks.
We offer an extensive review of over 60 HTL methods, covering both data-based and model-based approaches.
We explore applications in natural language processing, computer vision, multimodal learning, and biomedicine.
arXiv Detail & Related papers (2023-10-12T16:19:58Z) - Iterative Zero-Shot LLM Prompting for Knowledge Graph Construction [104.29108668347727]
This paper proposes an innovative knowledge graph generation approach that leverages the potential of the latest generative large language models.
The approach is conveyed in a pipeline that comprises novel iterative zero-shot and external knowledge-agnostic strategies.
We claim that our proposal is a suitable solution for scalable and versatile knowledge graph construction and may be applied to different and novel contexts.
arXiv Detail & Related papers (2023-07-03T16:01:45Z) - Foundations and Recent Trends in Multimodal Machine Learning:
Principles, Challenges, and Open Questions [68.6358773622615]
This paper provides an overview of the computational and theoretical foundations of multimodal machine learning.
We propose a taxonomy of 6 core technical challenges: representation, alignment, reasoning, generation, transference, and quantification.
Recent technical achievements will be presented through the lens of this taxonomy, allowing researchers to understand the similarities and differences across new approaches.
arXiv Detail & Related papers (2022-09-07T19:21:19Z) - Behavior Priors for Efficient Reinforcement Learning [97.81587970962232]
We consider how information and architectural constraints can be combined with ideas from the probabilistic modeling literature to learn behavior priors.
We discuss how such latent variable formulations connect to related work on hierarchical reinforcement learning (HRL) and mutual information and curiosity based objectives.
We demonstrate the effectiveness of our framework by applying it to a range of simulated continuous control domains.
arXiv Detail & Related papers (2020-10-27T13:17:18Z) - Uniform Priors for Data-Efficient Transfer [65.086680950871]
We show that features that are most transferable have high uniformity in the embedding space.
We evaluate the regularization on its ability to facilitate adaptation to unseen tasks and data.
arXiv Detail & Related papers (2020-06-30T04:39:36Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.