Task Discovery: Finding the Tasks that Neural Networks Generalize on
- URL: http://arxiv.org/abs/2212.00261v1
- Date: Thu, 1 Dec 2022 03:57:48 GMT
- Title: Task Discovery: Finding the Tasks that Neural Networks Generalize on
- Authors: Andrei Atanov, Andrei Filatov, Teresa Yeo, Ajay Sohmshetty, Amir Zamir
- Abstract summary: We show that one set of images can give rise to many tasks on which neural networks generalize well.
As an example, we show that the discovered tasks can be used to automatically create adversarial train-test splits.
- Score: 1.4043229953691112
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: When developing deep learning models, we usually decide what task we want to
solve then search for a model that generalizes well on the task. An intriguing
question would be: what if, instead of fixing the task and searching in the
model space, we fix the model and search in the task space? Can we find tasks
that the model generalizes on? How do they look, or do they indicate anything?
These are the questions we address in this paper.
We propose a task discovery framework that automatically finds examples of
such tasks via optimizing a generalization-based quantity called agreement
score. We demonstrate that one set of images can give rise to many tasks on
which neural networks generalize well. These tasks are a reflection of the
inductive biases of the learning framework and the statistical patterns present
in the data, thus they can make a useful tool for analysing the neural networks
and their biases. As an example, we show that the discovered tasks can be used
to automatically create adversarial train-test splits which make a model fail
at test time, without changing the pixels or labels, but by only selecting how
the datapoints should be split between the train and test sets. We end with a
discussion on human-interpretability of the discovered tasks.
Related papers
- Finding Visual Task Vectors [74.67336516908776]
Visual Prompting is a technique for teaching models to perform a visual task via in-context examples, without any additional training.
We analyze the activations of MAE-VQGAN, a recent Visual Prompting model, and find task vectors, activations that encode task-specific information.
arXiv Detail & Related papers (2024-04-08T17:59:46Z) - Provable Multi-Task Representation Learning by Two-Layer ReLU Neural Networks [69.38572074372392]
We present the first results proving that feature learning occurs during training with a nonlinear model on multiple tasks.
Our key insight is that multi-task pretraining induces a pseudo-contrastive loss that favors representations that align points that typically have the same label across tasks.
arXiv Detail & Related papers (2023-07-13T16:39:08Z) - Are Deep Neural Networks SMARTer than Second Graders? [85.60342335636341]
We evaluate the abstraction, deduction, and generalization abilities of neural networks in solving visuo-linguistic puzzles designed for children in the 6--8 age group.
Our dataset consists of 101 unique puzzles; each puzzle comprises a picture question, and their solution needs a mix of several elementary skills, including arithmetic, algebra, and spatial reasoning.
Experiments reveal that while powerful deep models offer reasonable performances on puzzles in a supervised setting, they are not better than random accuracy when analyzed for generalization.
arXiv Detail & Related papers (2022-12-20T04:33:32Z) - Task Compass: Scaling Multi-task Pre-training with Task Prefix [122.49242976184617]
Existing studies show that multi-task learning with large-scale supervised tasks suffers from negative effects across tasks.
We propose a task prefix guided multi-task pre-training framework to explore the relationships among tasks.
Our model can not only serve as the strong foundation backbone for a wide range of tasks but also be feasible as a probing tool for analyzing task relationships.
arXiv Detail & Related papers (2022-10-12T15:02:04Z) - Backbones-Review: Feature Extraction Networks for Deep Learning and Deep
Reinforcement Learning Approaches [3.255610188565679]
CNNs allow to work on large-scale size of data, as well as cover different scenarios for a specific task.
Many networks have been proposed and become the famous networks used for any DL models in any AI task.
A backbone is a known network trained in many other tasks before and demonstrates its effectiveness.
arXiv Detail & Related papers (2022-06-16T09:18:34Z) - Using a Cross-Task Grid of Linear Probes to Interpret CNN Model
Predictions On Retinal Images [3.5789352263336847]
We analyze a dataset of retinal images using linear probes: linear regression models trained on some "target" task, using embeddings from a deep convolutional (CNN) model trained on some "source" task as input.
We use this method across all possible pairings of 93 tasks in the UK Biobank dataset of retinal images, leading to 164k different models.
arXiv Detail & Related papers (2021-07-23T21:30:27Z) - Distribution Matching for Heterogeneous Multi-Task Learning: a
Large-scale Face Study [75.42182503265056]
Multi-Task Learning has emerged as a methodology in which multiple tasks are jointly learned by a shared learning algorithm.
We deal with heterogeneous MTL, simultaneously addressing detection, classification & regression problems.
We build FaceBehaviorNet, the first framework for large-scale face analysis, by jointly learning all facial behavior tasks.
arXiv Detail & Related papers (2021-05-08T22:26:52Z) - One Network Fits All? Modular versus Monolithic Task Formulations in
Neural Networks [36.07011014271394]
We show that a single neural network is capable of simultaneously learning multiple tasks from a combined data set.
We study how the complexity of learning such combined tasks grows with the complexity of the task codes.
arXiv Detail & Related papers (2021-03-29T01:16:42Z) - Assisting Scene Graph Generation with Self-Supervision [21.89909688056478]
We propose a set of three novel yet simple self-supervision tasks and train them as auxiliary multi-tasks to the main model.
While comparing, we train the base-model from scratch with these self-supervision tasks, we achieve state-of-the-art results in all the metrics and recall settings.
arXiv Detail & Related papers (2020-08-08T16:38:03Z) - Adaptive Task Sampling for Meta-Learning [79.61146834134459]
Key idea of meta-learning for few-shot classification is to mimic the few-shot situations faced at test time.
We propose an adaptive task sampling method to improve the generalization performance.
arXiv Detail & Related papers (2020-07-17T03:15:53Z) - Conditional Channel Gated Networks for Task-Aware Continual Learning [44.894710899300435]
Convolutional Neural Networks experience catastrophic forgetting when optimized on a sequence of learning problems.
We introduce a novel framework to tackle this problem with conditional computation.
We validate our proposal on four continual learning datasets.
arXiv Detail & Related papers (2020-03-31T19:35:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.