Towards Lifelong Self-Supervision For Unpaired Image-to-Image Translation

• URL: http://arxiv.org/abs/2004.00161v1
• Date: Tue, 31 Mar 2020 23:23:51 GMT
• Title: Towards Lifelong Self-Supervision For Unpaired Image-to-Image Translation
• Authors: Victor Schmidt, Makesh Narsimhan Sreedhar, Mostafa ElAraby, Irina Rish
• Abstract summary: We introduce Lifelong Self-Supervision (LiSS) as a way to pre-train an I2IT model on a set of self-supervised auxiliary tasks. We show that models trained with LiSS perform better on past tasks, while also being more robust than the CycleGAN baseline to color bias and entity entanglement.
• Score: 8.597098266150295
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Unpaired Image-to-Image Translation (I2IT) tasks often suffer from lack of data, a problem which self-supervised learning (SSL) has recently been very popular and successful at tackling. Leveraging auxiliary tasks such as rotation prediction or generative colorization, SSL can produce better and more robust representations in a low data regime. Training such tasks along an I2IT task is however computationally intractable as model size and the number of task grow. On the other hand, learning sequentially could incur catastrophic forgetting of previously learned tasks. To alleviate this, we introduce Lifelong Self-Supervision (LiSS) as a way to pre-train an I2IT model (e.g., CycleGAN) on a set of self-supervised auxiliary tasks. By keeping an exponential moving average of past encoders and distilling the accumulated knowledge, we are able to maintain the network's validation performance on a number of tasks without any form of replay, parameter isolation or retraining techniques typically used in continual learning. We show that models trained with LiSS perform better on past tasks, while also being more robust than the CycleGAN baseline to color bias and entity entanglement (when two entities are very close).

Related papers

• Continual Diffuser (CoD): Mastering Continual Offline Reinforcement Learning with Experience Rehearsal [54.93261535899478]
  In real-world applications, such as robotic control of reinforcement learning, the tasks are changing, and new tasks arise in a sequential order. This situation poses the new challenge of plasticity-stability trade-off for training an agent who can adapt to task changes and retain acquired knowledge. We propose a rehearsal-based continual diffusion model, called Continual diffuser (CoD), to endow the diffuser with the capabilities of quick adaptation (plasticity) and lasting retention (stability)
  arXiv  Detail & Related papers  (2024-09-04T08:21:47Z)
• Controlling Forgetting with Test-Time Data in Continual Learning [15.455400390299593]
  Ongoing Continual Learning research provides techniques to overcome catastrophic forgetting of previous information when new knowledge is acquired. We argue that test-time data hold great information that can be leveraged in a self supervised manner to refresh the model's memory of previous learned tasks.
  arXiv  Detail & Related papers  (2024-06-19T15:56:21Z)
• Provable Multi-Task Representation Learning by Two-Layer ReLU Neural Networks [69.38572074372392]
  We present the first results proving that feature learning occurs during training with a nonlinear model on multiple tasks. Our key insight is that multi-task pretraining induces a pseudo-contrastive loss that favors representations that align points that typically have the same label across tasks.
  arXiv  Detail & Related papers  (2023-07-13T16:39:08Z)
• Preventing Catastrophic Forgetting in Continual Learning of New Natural Language Tasks [17.879087904904935]
  Multi-Task Learning (MTL) is widely-accepted in Natural Language Processing as a standard technique for learning multiple related tasks in one model. As systems usually evolve over time, adding a new task to an existing MTL model usually requires retraining the model from scratch on all the tasks. In this paper, we approach the problem of incrementally expanding MTL models' capability to solve new tasks over time by distilling the knowledge of an already trained model on n tasks into a new one for solving n+1 tasks.
  arXiv  Detail & Related papers  (2023-02-22T00:18:25Z)
• Learn from Yesterday: A Semi-Supervised Continual Learning Method for Supervision-Limited Text-to-SQL Task Streams [18.010095381310972]
  This paper proposes integrating semi-supervised learning (SSL) and continual learning (CL) in a stream of text-to-labeled tasks. The experiments on two datasets shows that SFNet outperforms the widely-used SSL-only and CL-only baselines on multiple metrics.
  arXiv  Detail & Related papers  (2022-11-21T07:40:28Z)
• Task Compass: Scaling Multi-task Pre-training with Task Prefix [122.49242976184617]
  Existing studies show that multi-task learning with large-scale supervised tasks suffers from negative effects across tasks. We propose a task prefix guided multi-task pre-training framework to explore the relationships among tasks. Our model can not only serve as the strong foundation backbone for a wide range of tasks but also be feasible as a probing tool for analyzing task relationships.
  arXiv  Detail & Related papers  (2022-10-12T15:02:04Z)
• Task-Customized Self-Supervised Pre-training with Scalable Dynamic Routing [76.78772372631623]
  A common practice for self-supervised pre-training is to use as much data as possible. For a specific downstream task, however, involving irrelevant data in pre-training may degenerate the downstream performance. It is burdensome and infeasible to use different downstream-task-customized datasets in pre-training for different tasks.
  arXiv  Detail & Related papers  (2022-05-26T10:49:43Z)
• Hierarchical Few-Shot Imitation with Skill Transition Models [66.81252581083199]
  Few-shot Imitation with Skill Transition Models (FIST) is an algorithm that extracts skills from offline data and utilizes them to generalize to unseen tasks. We show that FIST is capable of generalizing to new tasks and substantially outperforms prior baselines in navigation experiments.
  arXiv  Detail & Related papers  (2021-07-19T15:56:01Z)
• Adversarial Training of Variational Auto-encoders for Continual Zero-shot Learning [1.90365714903665]
  We present a hybrid network that consists of a shared VAE module to hold information of all tasks and task-specific private VAE modules for each task. The model's size grows with each task to prevent catastrophic forgetting of task-specific skills. We show our method is superior on class sequentially learning with ZSL(Zero-Shot Learning) and GZSL(Generalized Zero-Shot Learning)
  arXiv  Detail & Related papers  (2021-02-07T11:21:24Z)
• Representation Ensembling for Synergistic Lifelong Learning with Quasilinear Complexity [17.858926093389737]
  In lifelong learning, data are used to improve performance not only on the current task, but also on previously encountered, and as yet unencountered tasks. Our key insight is that we can synergistically ensemble representations -- that were learned independently on disparate tasks -- to enable both forward and backward transfer.
  arXiv  Detail & Related papers  (2020-04-27T16:16:30Z)
• Generalized Hindsight for Reinforcement Learning [154.0545226284078]
  We argue that low-reward data collected while trying to solve one task provides little to no signal for solving that particular task. We present Generalized Hindsight: an approximate inverse reinforcement learning technique for relabeling behaviors with the right tasks.
  arXiv  Detail & Related papers  (2020-02-26T18:57:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.