Knowledge Transfer for Cross-Domain Reinforcement Learning: A Systematic Review

• URL: http://arxiv.org/abs/2404.17687v1
• Date: Fri, 26 Apr 2024 20:36:58 GMT
• Title: Knowledge Transfer for Cross-Domain Reinforcement Learning: A Systematic Review
• Authors: Sergio A. Serrano, Jose Martinez-Carranza, L. Enrique Sucar,
• Abstract summary: Reinforcement Learning (RL) provides a framework in which agents can be trained, via trial and error, to solve complex decision-making problems. By reusing knowledge from a different task, knowledge transfer methods present an alternative to reduce the training time in RL. This review presents a unifying analysis of methods focused on transferring knowledge across different domains.
• Score: 2.94944680995069
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Reinforcement Learning (RL) provides a framework in which agents can be trained, via trial and error, to solve complex decision-making problems. Learning with little supervision causes RL methods to require large amounts of data, which renders them too expensive for many applications (e.g. robotics). By reusing knowledge from a different task, knowledge transfer methods present an alternative to reduce the training time in RL. Given how severe data scarcity can be, there has been a growing interest for methods capable of transferring knowledge across different domains (i.e. problems with different representation) due to the flexibility they offer. This review presents a unifying analysis of methods focused on transferring knowledge across different domains. Through a taxonomy based on a transfer-approach categorization, and a characterization of works based on their data-assumption requirements, the objectives of this article are to 1) provide a comprehensive and systematic revision of knowledge transfer methods for the cross-domain RL setting, 2) categorize and characterize these methods to provide an analysis based on relevant features such as their transfer approach and data requirements, and 3) discuss the main challenges regarding cross-domain knowledge transfer, as well as ideas of future directions worth exploring to address these problems.

Related papers

• CrowdTransfer: Enabling Crowd Knowledge Transfer in AIoT Community [12.002871068635748]
  Crowd Knowledge Transfer (CrowdTransfer) aims to transfer prior knowledge learned from a crowd of agents to reduce the training cost. We present four transfer modes from the perspective of crowd intelligence, including derivation, sharing, evolution and fusion modes. We explore some applications of AIoT areas, such as human activity recognition, urban computing, multi-robot system, and smart factory.
  arXiv  Detail & Related papers  (2024-07-09T01:20:37Z)
• Bayesian Transfer Learning [13.983016833412307]
  "Transfer learning" seeks to improve inference and/or predictive accuracy on a domain of interest by leveraging data from related domains. This article highlights Bayesian approaches to transfer learning, which have received relatively limited attention despite their innate compatibility with the notion of drawing upon prior knowledge to guide new learning tasks. We discuss how these methods address the problem of finding the optimal information to transfer between domains, which is a central question in transfer learning.
  arXiv  Detail & Related papers  (2023-12-20T23:38:17Z)
• A Recent Survey of Heterogeneous Transfer Learning [15.830786437956144]
  heterogeneous transfer learning has become a vital strategy in various tasks. We offer an extensive review of over 60 HTL methods, covering both data-based and model-based approaches. We explore applications in natural language processing, computer vision, multimodal learning, and biomedicine.
  arXiv  Detail & Related papers  (2023-10-12T16:19:58Z)
• A Study of Situational Reasoning for Traffic Understanding [63.45021731775964]
  We devise three novel text-based tasks for situational reasoning in the traffic domain. We adopt four knowledge-enhanced methods that have shown generalization capability across language reasoning tasks in prior work. We provide in-depth analyses of model performance on data partitions and examine model predictions categorically.
  arXiv  Detail & Related papers  (2023-06-05T01:01:12Z)
• Graph Enabled Cross-Domain Knowledge Transfer [1.52292571922932]
  Cross-Domain Knowledge Transfer is an approach to mitigate the gap between good representation learning and the scarce knowledge in the domain of interest. From the machine learning perspective, the paradigm of semi-supervised learning takes advantage of large amount of data without ground truth and achieves impressive learning performance improvement.
  arXiv  Detail & Related papers  (2023-04-07T03:02:10Z)
• Multi-Source Transfer Learning for Deep Model-Based Reinforcement Learning [0.6445605125467572]
  A crucial challenge in reinforcement learning is to reduce the number of interactions with the environment that an agent requires to master a given task. Transfer learning proposes to address this issue by re-using knowledge from previously learned tasks. The goal of this paper is to address these issues with modular multi-source transfer learning techniques.
  arXiv  Detail & Related papers  (2022-05-28T12:04:52Z)
• Transferred Q-learning [79.79659145328856]
  We consider $Q$-learning with knowledge transfer, using samples from a target reinforcement learning (RL) task as well as source samples from different but related RL tasks. We propose transfer learning algorithms for both batch and online $Q$-learning with offline source studies.
  arXiv  Detail & Related papers  (2022-02-09T20:08:19Z)
• Transferability in Deep Learning: A Survey [80.67296873915176]
  The ability to acquire and reuse knowledge is known as transferability in deep learning. We present this survey to connect different isolated areas in deep learning with their relation to transferability. We implement a benchmark and an open-source library, enabling a fair evaluation of deep learning methods in terms of transferability.
  arXiv  Detail & Related papers  (2022-01-15T15:03:17Z)
• Towards Accurate Knowledge Transfer via Target-awareness Representation Disentanglement [56.40587594647692]
  We propose a novel transfer learning algorithm, introducing the idea of Target-awareness REpresentation Disentanglement (TRED) TRED disentangles the relevant knowledge with respect to the target task from the original source model and used as a regularizer during fine-tuning the target model. Experiments on various real world datasets show that our method stably improves the standard fine-tuning by more than 2% in average.
  arXiv  Detail & Related papers  (2020-10-16T17:45:08Z)
• What is being transferred in transfer learning? [51.6991244438545]
  We show that when training from pre-trained weights, the model stays in the same basin in the loss landscape. We present that when training from pre-trained weights, the model stays in the same basin in the loss landscape and different instances of such model are similar in feature space and close in parameter space.
  arXiv  Detail & Related papers  (2020-08-26T17:23:40Z)
• Provable Meta-Learning of Linear Representations [114.656572506859]
  We provide fast, sample-efficient algorithms to address the dual challenges of learning a common set of features from multiple, related tasks, and transferring this knowledge to new, unseen tasks. We also provide information-theoretic lower bounds on the sample complexity of learning these linear features.
  arXiv  Detail & Related papers  (2020-02-26T18:21:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.