Searching for Controllable Image Restoration Networks

• URL: http://arxiv.org/abs/2012.11225v1
• Date: Mon, 21 Dec 2020 10:08:18 GMT
• Title: Searching for Controllable Image Restoration Networks
• Authors: Heewon Kim, Sungyong Baik, Myungsub Choi, Janghoon Choi, Kyoung Mu Lee
• Abstract summary: Existing methods require separate inference through the entire network per each output. We propose a novel framework based on a neural architecture search technique that enables efficient generation of multiple imagery effects.
• Score: 57.23583915884236
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diverse user preferences over images have recently led to a great amount of interest in controlling the imagery effects for image restoration tasks. However, existing methods require separate inference through the entire network per each output, which hinders users from readily comparing multiple imagery effects due to long latency. To this end, we propose a novel framework based on a neural architecture search technique that enables efficient generation of multiple imagery effects via two stages of pruning: task-agnostic and task-specific pruning. Specifically, task-specific pruning learns to adaptively remove the irrelevant network parameters for each task, while task-agnostic pruning learns to find an efficient architecture by sharing the early layers of the network across different tasks. Since the shared layers allow for feature reuse, only a single inference of the task-agnostic layers is needed to generate multiple imagery effects from the input image. Using the proposed task-agnostic and task-specific pruning schemes together significantly reduces the FLOPs and the actual latency of inference compared to the baseline. We reduce 95.7% of the FLOPs when generating 27 imagery effects, and make the GPU latency 73.0% faster on 4K-resolution images.

Related papers

• A Point-Based Approach to Efficient LiDAR Multi-Task Perception [49.91741677556553]
  PAttFormer is an efficient multi-task architecture for joint semantic segmentation and object detection in point clouds. Unlike other LiDAR-based multi-task architectures, our proposed PAttFormer does not require separate feature encoders for task-specific point cloud representations. Our evaluations show substantial gains from multi-task learning, improving LiDAR semantic segmentation by +1.7% in mIou and 3D object detection by +1.7% in mAP.
  arXiv  Detail & Related papers  (2024-04-19T11:24:34Z)
• Real-Time Image Segmentation via Hybrid Convolutional-Transformer Architecture Search [49.81353382211113]
  We address the challenge of integrating multi-head self-attention into high resolution representation CNNs efficiently. We develop a multi-target multi-branch supernet method, which fully utilizes the advantages of high-resolution features. We present a series of model via Hybrid Convolutional-Transformer Architecture Search (HyCTAS) method that searched for the best hybrid combination of light-weight convolution layers and memory-efficient self-attention layers.
  arXiv  Detail & Related papers  (2024-03-15T15:47:54Z)
• Unified-Width Adaptive Dynamic Network for All-In-One Image Restoration [50.81374327480445]
  We introduce a novel concept positing that intricate image degradation can be represented in terms of elementary degradation. We propose the Unified-Width Adaptive Dynamic Network (U-WADN), consisting of two pivotal components: a Width Adaptive Backbone (WAB) and a Width Selector (WS) The proposed U-WADN achieves better performance while simultaneously reducing up to 32.3% of FLOPs and providing approximately 15.7% real-time acceleration.
  arXiv  Detail & Related papers  (2024-01-24T04:25:12Z)
• Towards Efficient Single Image Dehazing and Desnowing [6.052434200703146]
  We propose an efficient and compact image restoration network named DAN-Net to address this problem. A single expert network efficiently addresses specific degradation in nasty winter scenes relying on the compact architecture and three novel components. We have collected the first real-world winter scenes dataset to evaluate winter image restoration methods.
  arXiv  Detail & Related papers  (2022-04-19T13:51:02Z)
• Leveraging Image Complexity in Macro-Level Neural Network Design for Medical Image Segmentation [3.974175960216864]
  We show that image complexity can be used as a guideline in choosing what is best for a given dataset. For high-complexity datasets, a shallow network running on the original images may yield better segmentation results than a deep network running on downsampled images.
  arXiv  Detail & Related papers  (2021-12-21T09:49:47Z)
• Hierarchical Image Peeling: A Flexible Scale-space Filtering Framework [0.0]
  hierarchical image organization has been witnessed by a wide spectrum of applications in computer vision and graphics. This work designs a modern framework for disassembling an image into a family of derived signals from a scale-space perspective. A compact recurrent network, namely hierarchical image peeling net, is customized to efficiently and effectively fulfill the task.
  arXiv  Detail & Related papers  (2021-04-04T04:08:14Z)
• Powers of layers for image-to-image translation [60.5529622990682]
  We propose a simple architecture to address unpaired image-to-image translation tasks. We start from an image autoencoder architecture with fixed weights. For each task we learn a residual block operating in the latent space, which is iteratively called until the target domain is reached.
  arXiv  Detail & Related papers  (2020-08-13T09:02:17Z)
• Deeply Cascaded U-Net for Multi-Task Image Processing [1.3815111881580258]
  We propose a novel multi-task neural network architecture designed for combining sequential image processing tasks. We extend U-Net by additional decoding pathways for each individual task, and explore deep cascading of outputs and connectivity from one pathway to another.
  arXiv  Detail & Related papers  (2020-05-01T05:06:35Z)
• CRNet: Cross-Reference Networks for Few-Shot Segmentation [59.85183776573642]
  Few-shot segmentation aims to learn a segmentation model that can be generalized to novel classes with only a few training images. With a cross-reference mechanism, our network can better find the co-occurrent objects in the two images. Experiments on the PASCAL VOC 2012 dataset show that our network achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-03-24T04:55:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.