Cross-Stitched Multi-task Dual Recursive Networks for Unified Single Image Deraining and Desnowing

• URL: http://arxiv.org/abs/2211.08290v1
• Date: Tue, 15 Nov 2022 16:44:53 GMT
• Title: Cross-Stitched Multi-task Dual Recursive Networks for Unified Single Image Deraining and Desnowing
• Authors: Sotiris Karavarsamis, Alexandros Doumanoglou, Konstantinos Konstantoudakis, Dimitrios Zarpalas
• Abstract summary: We present the Cross-stitched Multi-task Unified Dual Recursive Network (CMUDRN) model targeting the task of unified deraining and desnowing. The proposed model makes use of cross-stitch units that enable multi-task learning across two separate Dual Recursive Network (DRN) models.
• Score: 70.24489870383027
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We present the Cross-stitched Multi-task Unified Dual Recursive Network (CMUDRN) model targeting the task of unified deraining and desnowing in a multi-task learning setting. This unified model borrows from the basic Dual Recursive Network (DRN) architecture developed by Cai et al. The proposed model makes use of cross-stitch units that enable multi-task learning across two separate DRN models, each tasked for single image deraining and desnowing, respectively. By fixing cross-stitch units at several layers of basic task-specific DRN networks, we perform multi-task learning over the two separate DRN models. To enable blind image restoration, on top of these structures we employ a simple neural fusion scheme which merges the output of each DRN. The separate task-specific DRN models and the fusion scheme are simultaneously trained by enforcing local and global supervision. Local supervision is applied on the two DRN submodules, and global supervision is applied on the data fusion submodule of the proposed model. Consequently, we both enable feature sharing across task-specific DRN models and control the image restoration behavior of the DRN submodules. An ablation study shows the strength of the hypothesized CMUDRN model, and experiments indicate that its performance is comparable or better than baseline DRN models on the single image deraining and desnowing tasks. Moreover, CMUDRN enables blind image restoration for the two underlying image restoration tasks, by unifying task-specific image restoration pipelines via a naive parametric fusion scheme. The CMUDRN implementation is available at https://github.com/VCL3D/CMUDRN.

Related papers

• Representation Surgery for Multi-Task Model Merging [57.63643005215592]
  Multi-task learning (MTL) compresses the information from multiple tasks into a unified backbone to improve computational efficiency and generalization. Recent work directly merges multiple independently trained models to perform MTL instead of collecting their raw data for joint training. By visualizing the representation distribution of existing model merging schemes, we find that the merged model often suffers from the dilemma of representation bias.
  arXiv  Detail & Related papers  (2024-02-05T03:39:39Z)
• Multi-task Image Restoration Guided By Robust DINO Features [88.74005987908443]
  We propose mboxtextbfDINO-IR, a multi-task image restoration approach leveraging robust features extracted from DINOv2. We first propose a pixel-semantic fusion (PSF) module to dynamically fuse DINOV2's shallow features. By formulating these modules into a unified deep model, we propose a DINO perception contrastive loss to constrain the model training.
  arXiv  Detail & Related papers  (2023-12-04T06:59:55Z)
• MSINet: Twins Contrastive Search of Multi-Scale Interaction for Object ReID [29.13844433114534]
  We propose a novel Twins Contrastive Mechanism (TCM) to provide more appropriate supervision for ReID architecture search. TCM reduces the category overlaps between the training and validation data, and assists NAS in simulating real-world ReID training schemes. We then design a Multi-Scale Interaction (MSI) search space to search for rational interaction operations between multi-scale features.
  arXiv  Detail & Related papers  (2023-03-13T12:39:59Z)
• Deep Image Clustering with Contrastive Learning and Multi-scale Graph Convolutional Networks [58.868899595936476]
  This paper presents a new deep clustering approach termed image clustering with contrastive learning and multi-scale graph convolutional networks (IcicleGCN) Experiments on multiple image datasets demonstrate the superior clustering performance of IcicleGCN over the state-of-the-art.
  arXiv  Detail & Related papers  (2022-07-14T19:16:56Z)
• Neural Implicit Dictionary via Mixture-of-Expert Training [111.08941206369508]
  We present a generic INR framework that achieves both data and training efficiency by learning a Neural Implicit Dictionary (NID) Our NID assembles a group of coordinate-based Impworks which are tuned to span the desired function space. Our experiments show that, NID can improve reconstruction of 2D images or 3D scenes by 2 orders of magnitude faster with up to 98% less input data.
  arXiv  Detail & Related papers  (2022-07-08T05:07:19Z)
• Single Image Dehazing with An Independent Detail-Recovery Network [117.86146907611054]
  We propose a single image dehazing method with an independent Detail Recovery Network (DRN) The DRN aims to recover the dehazed image details through local and global branches respectively. Our method outperforms the state-of-the-art dehazing methods both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2021-09-22T02:49:43Z)
• One Network to Solve Them All: A Sequential Multi-Task Joint Learning Network Framework for MR Imaging Pipeline [12.684219884940056]
  A sequential multi-task joint learning network model is proposed to train a combined end-to-end pipeline. The proposed framework is verified on MRB dataset, which achieves superior performance on other SOTA methods in terms of both reconstruction and segmentation.
  arXiv  Detail & Related papers  (2021-05-14T05:55:27Z)
• MDCN: Multi-scale Dense Cross Network for Image Super-Resolution [35.59973281360541]
  We propose a Multi-scale Dense Cross Network (MDCN), which achieves great performance with fewer parameters and less execution time. MDCN consists of multi-scale dense cross blocks (MDCBs), hierarchical feature distillation block (HFDB), and dynamic reconstruction block (DRB)
  arXiv  Detail & Related papers  (2020-08-30T03:50:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.