Composing Task Knowledge with Modular Successor Feature Approximators
- URL: http://arxiv.org/abs/2301.12305v2
- Date: Fri, 25 Aug 2023 18:28:51 GMT
- Title: Composing Task Knowledge with Modular Successor Feature Approximators
- Authors: Wilka Carvalho, Angelos Filos, Richard L. Lewis, Honglak lee, and
Satinder Singh
- Abstract summary: We present a novel neural network architecture, "Modular Successor Feature Approximators" (MSFA)
MSFA is able to better generalize compared to baseline architectures for learning SFs and modular architectures.
- Score: 60.431769158952626
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Recently, the Successor Features and Generalized Policy Improvement (SF&GPI)
framework has been proposed as a method for learning, composing, and
transferring predictive knowledge and behavior. SF&GPI works by having an agent
learn predictive representations (SFs) that can be combined for transfer to new
tasks with GPI. However, to be effective this approach requires state features
that are useful to predict, and these state-features are typically
hand-designed. In this work, we present a novel neural network architecture,
"Modular Successor Feature Approximators" (MSFA), where modules both discover
what is useful to predict, and learn their own predictive representations. We
show that MSFA is able to better generalize compared to baseline architectures
for learning SFs and modular architectures
Related papers
- Graph Foundation Models for Recommendation: A Comprehensive Survey [55.70529188101446]
Large language models (LLMs) are designed to process and comprehend natural language, making both approaches highly effective and widely adopted.
Recent research has focused on graph foundation models (GFMs)
GFMs integrate the strengths of GNNs and LLMs to model complex RS problems more efficiently by leveraging the graph-based structure of user-item relationships alongside textual understanding.
arXiv Detail & Related papers (2025-02-12T12:13:51Z) - SG-MIM: Structured Knowledge Guided Efficient Pre-training for Dense Prediction [17.44991827937427]
Masked Image Modeling techniques have redefined the landscape of computer vision.
Despite their success, the full potential of MIM-based methods in dense prediction tasks, particularly in depth estimation, remains untapped.
We propose SG-MIM, a novel Structured knowledge Guided Masked Image Modeling framework designed to enhance dense prediction tasks by utilizing structured knowledge alongside images.
arXiv Detail & Related papers (2024-09-04T08:24:53Z) - HPT++: Hierarchically Prompting Vision-Language Models with Multi-Granularity Knowledge Generation and Improved Structure Modeling [39.14392943549792]
We propose a novel approach called Hierarchical Prompt Tuning (HPT), enabling simultaneous modeling of both structured and conventional linguistic knowledge.
We introduce a relationship-guided attention module to capture pair-wise associations among entities and attributes for low-level prompt learning.
By incorporating high-level and global-level prompts modeling overall semantics, the proposed hierarchical structure forges cross-level interlinks and empowers the model to handle more complex and long-term relationships.
arXiv Detail & Related papers (2024-08-27T06:50:28Z) - On the verification of Embeddings using Hybrid Markov Logic [2.113770213797994]
We propose a framework to verify complex properties of a learned representation.
We present an approach to learn parameters for the properties within this framework.
We illustrate verification in Graph Neural Networks, Deep Knowledge Tracing and Intelligent Tutoring Systems.
arXiv Detail & Related papers (2023-12-13T17:04:09Z) - Combining Behaviors with the Successor Features Keyboard [55.983751286962985]
"Successor Features Keyboard" (SFK) enables transfer with discovered state-features and task encodings.
We achieve the first demonstration of transfer with SFs in a challenging 3D environment.
arXiv Detail & Related papers (2023-10-24T15:35:54Z) - An Expectation-Maximization Perspective on Federated Learning [75.67515842938299]
Federated learning describes the distributed training of models across multiple clients while keeping the data private on-device.
In this work, we view the server-orchestrated federated learning process as a hierarchical latent variable model where the server provides the parameters of a prior distribution over the client-specific model parameters.
We show that with simple Gaussian priors and a hard version of the well known Expectation-Maximization (EM) algorithm, learning in such a model corresponds to FedAvg, the most popular algorithm for the federated learning setting.
arXiv Detail & Related papers (2021-11-19T12:58:59Z) - Multi-Branch Deep Radial Basis Function Networks for Facial Emotion
Recognition [80.35852245488043]
We propose a CNN based architecture enhanced with multiple branches formed by radial basis function (RBF) units.
RBF units capture local patterns shared by similar instances using an intermediate representation.
We show it is the incorporation of local information what makes the proposed model competitive.
arXiv Detail & Related papers (2021-09-07T21:05:56Z) - How Fine-Tuning Allows for Effective Meta-Learning [50.17896588738377]
We present a theoretical framework for analyzing representations derived from a MAML-like algorithm.
We provide risk bounds on the best predictor found by fine-tuning via gradient descent, demonstrating that the algorithm can provably leverage the shared structure.
This separation result underscores the benefit of fine-tuning-based methods, such as MAML, over methods with "frozen representation" objectives in few-shot learning.
arXiv Detail & Related papers (2021-05-05T17:56:00Z) - Compositional Generalization in Semantic Parsing: Pre-training vs.
Specialized Architectures [1.8434042562191812]
We show that pre-training leads to significant improvements in performance vs. comparable non-pre-trained models.
We establish a new state of the art on the CFQ compositional generalization benchmark using pre-training together with an intermediate representation.
arXiv Detail & Related papers (2020-07-17T13:34:49Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.