AdaIR: Exploiting Underlying Similarities of Image Restoration Tasks with Adapters

• URL: http://arxiv.org/abs/2404.11475v1
• Date: Wed, 17 Apr 2024 15:31:06 GMT
• Title: AdaIR: Exploiting Underlying Similarities of Image Restoration Tasks with Adapters
• Authors: Hao-Wei Chen, Yu-Syuan Xu, Kelvin C. K. Chan, Hsien-Kai Kuo, Chun-Yi Lee, Ming-Hsuan Yang,
• Abstract summary: AdaIR is a novel framework that enables low storage cost and efficient training without sacrificing performance. AdaIR requires solely the training of lightweight, task-specific modules, ensuring a more efficient storage and training regimen.
• Score: 57.62742271140852
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Existing image restoration approaches typically employ extensive networks specifically trained for designated degradations. Despite being effective, such methods inevitably entail considerable storage costs and computational overheads due to the reliance on task-specific networks. In this work, we go beyond this well-established framework and exploit the inherent commonalities among image restoration tasks. The primary objective is to identify components that are shareable across restoration tasks and augment the shared components with modules specifically trained for individual tasks. Towards this goal, we propose AdaIR, a novel framework that enables low storage cost and efficient training without sacrificing performance. Specifically, a generic restoration network is first constructed through self-supervised pre-training using synthetic degradations. Subsequent to the pre-training phase, adapters are trained to adapt the pre-trained network to specific degradations. AdaIR requires solely the training of lightweight, task-specific modules, ensuring a more efficient storage and training regimen. We have conducted extensive experiments to validate the effectiveness of AdaIR and analyze the influence of the pre-training strategy on discovering shareable components. Extensive experimental results show that AdaIR achieves outstanding results on multi-task restoration while utilizing significantly fewer parameters (1.9 MB) and less training time (7 hours) for each restoration task. The source codes and trained models will be released.

Related papers

• LoRA-IR: Taming Low-Rank Experts for Efficient All-in-One Image Restoration [62.3751291442432]
  We propose LoRA-IR, a flexible framework that dynamically leverages compact low-rank experts to facilitate efficient all-in-one image restoration. LoRA-IR consists of two training stages: degradation-guided pre-training and parameter-efficient fine-tuning. LoRA-IR achieves state-of-the-art performance across 14 image restoration tasks and 29 benchmarks.
  arXiv  Detail & Related papers  (2024-10-20T13:00:24Z)
• Make the Pertinent Salient: Task-Relevant Reconstruction for Visual Control with Distractions [14.274653873720334]
  We propose a simple yet effective auxiliary task to facilitate representation learning in distracting environments. Under the assumption that task-relevant components of image observations are straightforward to identify with prior knowledge, we use a segmentation mask on image observations to only task-relevant components. In modified DeepMind Control suite (DMC) and Meta-World tasks with added visual distractions, SD achieves significantly better sample efficiency and greater final performance than prior work.
  arXiv  Detail & Related papers  (2024-10-13T19:24:07Z)
• MOST: MR reconstruction Optimization for multiple downStream Tasks via continual learning [12.0749219807816]
  Cascading separately trained reconstruction network and downstream task network has been shown to introduce performance degradation. We extend this optimization to sequentially introduced multiple downstream tasks and demonstrate that a single MR reconstruction network can be optimized for multiple downstream tasks.
  arXiv  Detail & Related papers  (2024-09-16T15:31:04Z)
• Boosting Image Restoration via Priors from Pre-trained Models [54.83907596825985]
  We learn an additional lightweight module called Pre-Train-Guided Refinement Module (PTG-RM) to refine restoration results of a target restoration network with OSF. PTG-RM effectively enhances restoration performance of various models across different tasks, including low-light enhancement, deraining, deblurring, and denoising.
  arXiv  Detail & Related papers  (2024-03-11T15:11:57Z)
• Meta-Learning with Versatile Loss Geometries for Fast Adaptation Using Mirror Descent [44.56938629818211]
  A fundamental challenge in meta-learning is how to quickly "adapt" the extracted prior in order to train a task-specific model. Existing approaches deal with this challenge using a preconditioner that enhances convergence of the per-task training process. The present contribution addresses this limitation by learning a nonlinear mirror map, which induces a versatile distance metric.
  arXiv  Detail & Related papers  (2023-12-20T23:45:06Z)
• Effective Adaptation in Multi-Task Co-Training for Unified Autonomous Driving [103.745551954983]
  In this paper, we investigate the transfer performance of various types of self-supervised methods, including MoCo and SimCLR, on three downstream tasks. We find that their performances are sub-optimal or even lag far behind the single-task baseline. We propose a simple yet effective pretrain-adapt-finetune paradigm for general multi-task training.
  arXiv  Detail & Related papers  (2022-09-19T12:15:31Z)
• E2-AEN: End-to-End Incremental Learning with Adaptively Expandable Network [57.87240860624937]
  We propose an end-to-end trainable adaptively expandable network named E2-AEN. It dynamically generates lightweight structures for new tasks without any accuracy drop in previous tasks. E2-AEN reduces cost and can be built upon any feed-forward architectures in an end-to-end manner.
  arXiv  Detail & Related papers  (2022-07-14T09:04:51Z)
• Proper Reuse of Image Classification Features Improves Object Detection [4.240984948137734]
  A common practice in transfer learning is to initialize the downstream model weights by pre-training on a data-abundant upstream task. Recent works show this is not strictly necessary under longer training regimes and provide recipes for training the backbone from scratch. We show that an extreme form of knowledge preservation -- freezing the classifier-d backbone -- consistently improves many different detection models.
  arXiv  Detail & Related papers  (2022-04-01T14:44:47Z)
• Rapid Structural Pruning of Neural Networks with Set-based Task-Adaptive Meta-Pruning [83.59005356327103]
  A common limitation of most existing pruning techniques is that they require pre-training of the network at least once before pruning. We propose STAMP, which task-adaptively prunes a network pretrained on a large reference dataset by generating a pruning mask on it as a function of the target dataset. We validate STAMP against recent advanced pruning methods on benchmark datasets.
  arXiv  Detail & Related papers  (2020-06-22T10:57:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.