Teaching with Commentaries
- URL: http://arxiv.org/abs/2011.03037v2
- Date: Fri, 12 Mar 2021 00:37:38 GMT
- Title: Teaching with Commentaries
- Authors: Aniruddh Raghu, Maithra Raghu, Simon Kornblith, David Duvenaud,
Geoffrey Hinton
- Abstract summary: We propose a flexible teaching framework using commentaries and learned meta-information.
We find that commentaries can improve training speed and/or performance.
commentaries can be reused when training new models to obtain performance benefits.
- Score: 108.62722733649542
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Effective training of deep neural networks can be challenging, and there
remain many open questions on how to best learn these models. Recently
developed methods to improve neural network training examine teaching:
providing learned information during the training process to improve downstream
model performance. In this paper, we take steps towards extending the scope of
teaching. We propose a flexible teaching framework using commentaries, learned
meta-information helpful for training on a particular task. We present
gradient-based methods to learn commentaries, leveraging recent work on
implicit differentiation for scalability. We explore diverse applications of
commentaries, from weighting training examples, to parameterising
label-dependent data augmentation policies, to representing attention masks
that highlight salient image regions. We find that commentaries can improve
training speed and/or performance, and provide insights about the dataset and
training process. We also observe that commentaries generalise: they can be
reused when training new models to obtain performance benefits, suggesting a
use-case where commentaries are stored with a dataset and leveraged in future
for improved model training.
Related papers
- Vision-Language Models Provide Promptable Representations for Reinforcement Learning [67.40524195671479]
We propose a novel approach that uses the vast amounts of general and indexable world knowledge encoded in vision-language models (VLMs) pre-trained on Internet-scale data for embodied reinforcement learning (RL)
We show that our approach can use chain-of-thought prompting to produce representations of common-sense semantic reasoning, improving policy performance in novel scenes by 1.5 times.
arXiv Detail & Related papers (2024-02-05T00:48:56Z) - Value Explicit Pretraining for Learning Transferable Representations [11.069853883599102]
We propose a method that learns generalizable representations for transfer reinforcement learning.
We learn new tasks that share similar objectives as previously learned tasks, by learning an encoder for objective-conditioned representations.
Experiments using a realistic navigation simulator and Atari benchmark show that the pretrained encoder produced by our method outperforms current SoTA pretraining methods.
arXiv Detail & Related papers (2023-12-19T17:12:35Z) - ALP: Action-Aware Embodied Learning for Perception [60.64801970249279]
We introduce Action-Aware Embodied Learning for Perception (ALP)
ALP incorporates action information into representation learning through a combination of optimizing a reinforcement learning policy and an inverse dynamics prediction objective.
We show that ALP outperforms existing baselines in several downstream perception tasks.
arXiv Detail & Related papers (2023-06-16T21:51:04Z) - PIVOT: Prompting for Video Continual Learning [50.80141083993668]
We introduce PIVOT, a novel method that leverages extensive knowledge in pre-trained models from the image domain.
Our experiments show that PIVOT improves state-of-the-art methods by a significant 27% on the 20-task ActivityNet setup.
arXiv Detail & Related papers (2022-12-09T13:22:27Z) - Learning New Skills after Deployment: Improving open-domain
internet-driven dialogue with human feedback [22.92577324751342]
We study how to improve internet-driven conversational skills in a learning framework.
We collect deployment data, and collect various types of human feedback.
We find the recently introduced Director model shows significant improvements over other existing approaches.
arXiv Detail & Related papers (2022-08-05T16:41:46Z) - (Un)likelihood Training for Interpretable Embedding [30.499562324921648]
Cross-modal representation learning has become a new normal for bridging the semantic gap between text and visual data.
We propose two novel training objectives, likelihood and unlikelihood functions, to unroll semantics behind embeddings.
With both training objectives, a new encoder-decoder network, which learns interpretable cross-modal representation, is proposed for ad-hoc video search.
arXiv Detail & Related papers (2022-07-01T09:15:02Z) - Reinforcement Learning with Action-Free Pre-Training from Videos [95.25074614579646]
We introduce a framework that learns representations useful for understanding the dynamics via generative pre-training on videos.
Our framework significantly improves both final performances and sample-efficiency of vision-based reinforcement learning.
arXiv Detail & Related papers (2022-03-25T19:44:09Z) - Interactive Machine Learning for Image Captioning [8.584932159968002]
We propose an approach for interactive learning for an image captioning model.
We envision a system that exploits human feedback as good as possible by multiplying the feedback using data augmentation methods.
arXiv Detail & Related papers (2022-02-28T09:02:32Z) - Learning Actor-centered Representations for Action Localization in
Streaming Videos using Predictive Learning [18.757368441841123]
Event perception tasks such as recognizing and localizing actions in streaming videos are essential for tackling visual understanding tasks.
We tackle the problem of learning textitactor-centered representations through the notion of continual hierarchical predictive learning.
Inspired by cognitive theories of event perception, we propose a novel, self-supervised framework.
arXiv Detail & Related papers (2021-04-29T06:06:58Z) - Self-supervised Co-training for Video Representation Learning [103.69904379356413]
We investigate the benefit of adding semantic-class positives to instance-based Info Noise Contrastive Estimation training.
We propose a novel self-supervised co-training scheme to improve the popular infoNCE loss.
We evaluate the quality of the learnt representation on two different downstream tasks: action recognition and video retrieval.
arXiv Detail & Related papers (2020-10-19T17:59:01Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.