FAST: Similarity-based Knowledge Transfer for Efficient Policy Learning

• URL: http://arxiv.org/abs/2507.20433v1
• Date: Sun, 27 Jul 2025 22:21:53 GMT
• Title: FAST: Similarity-based Knowledge Transfer for Efficient Policy Learning
• Authors: Alessandro Capurso, Elia Piccoli, Davide Bacciu,
• Abstract summary: Transfer Learning offers the potential to accelerate learning by transferring knowledge across tasks.<n>It faces critical challenges such as negative transfer, domain adaptation and inefficiency in selecting solid source policies.<n>In this work we challenge the key issues in TL to improve knowledge transfer, agents performance across tasks and reduce computational costs.
• Score: 57.4737157531239
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Transfer Learning (TL) offers the potential to accelerate learning by transferring knowledge across tasks. However, it faces critical challenges such as negative transfer, domain adaptation and inefficiency in selecting solid source policies. These issues often represent critical problems in evolving domains, i.e. game development, where scenarios transform and agents must adapt. The continuous release of new agents is costly and inefficient. In this work we challenge the key issues in TL to improve knowledge transfer, agents performance across tasks and reduce computational costs. The proposed methodology, called FAST - Framework for Adaptive Similarity-based Transfer, leverages visual frames and textual descriptions to create a latent representation of tasks dynamics, that is exploited to estimate similarity between environments. The similarity scores guides our method in choosing candidate policies from which transfer abilities to simplify learning of novel tasks. Experimental results, over multiple racing tracks, demonstrate that FAST achieves competitive final performance compared to learning-from-scratch methods while requiring significantly less training steps. These findings highlight the potential of embedding-driven task similarity estimations.

Related papers

• Enhancing Cross-task Transfer of Large Language Models via Activation Steering [75.41750053623298]
  Cross-task in-context learning offers a direct solution for transferring knowledge across tasks.<n>We investigate whether cross-task transfer can be achieved via latent space steering without parameter updates or input expansion.<n>We propose a novel Cross-task Activation Steering Transfer framework that enables effective transfer by manipulating the model's internal activation states.
  arXiv  Detail & Related papers  (2025-07-17T15:47:22Z)
• Exploring CausalWorld: Enhancing robotic manipulation via knowledge transfer and curriculum learning [6.683222869973898]
  This study explores a learning-based tri-finger robotic arm manipulating task, which requires complex movements and coordination among the fingers. By employing reinforcement learning, we train an agent to acquire the necessary skills for proficient manipulation. Two knowledge transfer strategies, fine-tuning and curriculum learning, were utilized within the soft actor-critic architecture.
  arXiv  Detail & Related papers  (2024-03-25T23:19:19Z)
• Feasibility of Transfer Learning: A Mathematical Framework [4.530876736231948]
  It begins by establishing the necessary mathematical concepts and constructing a mathematical framework for transfer learning. It then identifies and formulates the three-step transfer learning procedure as an optimization problem, allowing for the resolution of the feasibility issue.
  arXiv  Detail & Related papers  (2023-05-22T12:44:38Z)
• An Information-Theoretic Approach to Transferability in Task Transfer Learning [16.05523977032659]
  Task transfer learning is a popular technique in image processing applications that uses pre-trained models to reduce the supervision cost of related tasks. We present a novel metric, H-score, that estimates the performance of transferred representations from one task to another in classification problems.
  arXiv  Detail & Related papers  (2022-12-20T08:47:17Z)
• On Transferability of Prompt Tuning for Natural Language Understanding [63.29235426932978]
  We investigate the transferability of soft prompts across different tasks and models. We find that trained soft prompts can well transfer to similar tasks and initialize PT for them to accelerate training and improve performance. Our findings show that improving PT with knowledge transfer is possible and promising, while prompts' cross-task transferability is generally better than the cross-model transferability.
  arXiv  Detail & Related papers  (2021-11-12T13:39:28Z)
• Frustratingly Easy Transferability Estimation [64.42879325144439]
  We propose a simple, efficient, and effective transferability measure named TransRate. TransRate measures the transferability as the mutual information between the features of target examples extracted by a pre-trained model and labels of them. Despite its extraordinary simplicity in 10 lines of codes, TransRate performs remarkably well in extensive evaluations on 22 pre-trained models and 16 downstream tasks.
  arXiv  Detail & Related papers  (2021-06-17T10:27:52Z)
• Adaptive Policy Transfer in Reinforcement Learning [9.594432031144715]
  We introduce a principled mechanism that can "Adapt-to-Learn", that is adapt the source policy to learn to solve a target task. We show that the presented method learns to seamlessly combine learning from adaptation and exploration and leads to a robust policy transfer algorithm.
  arXiv  Detail & Related papers  (2021-05-10T22:42:03Z)
• Exploring and Predicting Transferability across NLP Tasks [115.6278033699853]
  We study the transferability between 33 NLP tasks across three broad classes of problems. Our results show that transfer learning is more beneficial than previously thought. We also develop task embeddings that can be used to predict the most transferable source tasks for a given target task.
  arXiv  Detail & Related papers  (2020-05-02T09:39:36Z)
• Transfer Heterogeneous Knowledge Among Peer-to-Peer Teammates: A Model Distillation Approach [55.83558520598304]
  We propose a brand new solution to reuse experiences and transfer value functions among multiple students via model distillation. We also describe how to design an efficient communication protocol to exploit heterogeneous knowledge. Our proposed framework, namely Learning and Teaching Categorical Reinforcement, shows promising performance on stabilizing and accelerating learning progress.
  arXiv  Detail & Related papers  (2020-02-06T11:31:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.