Discovering "Semantics" in Super-Resolution Networks

• URL: http://arxiv.org/abs/2108.00406v1
• Date: Sun, 1 Aug 2021 09:12:44 GMT
• Title: Discovering "Semantics" in Super-Resolution Networks
• Authors: Yihao Liu, Anran Liu, Jinjin Gu, Zhipeng Zhang, Wenhao Wu, Yu Qiao, Chao Dong
• Abstract summary: Super-resolution (SR) is a fundamental and representative task of low-level vision area. It is generally thought that the features extracted from the SR network have no specific semantic information. Can we find any "semantics" in SR networks?
• Score: 54.45509260681529
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Super-resolution (SR) is a fundamental and representative task of low-level vision area. It is generally thought that the features extracted from the SR network have no specific semantic information, and the network simply learns complex non-linear mappings from input to output. Can we find any "semantics" in SR networks? In this paper, we give affirmative answers to this question. By analyzing the feature representations with dimensionality reduction and visualization, we successfully discover the deep semantic representations in SR networks, \textit{i.e.}, deep degradation representations (DDR), which relate to the image degradation types and degrees. We also reveal the differences in representation semantics between classification and SR networks. Through extensive experiments and analysis, we draw a series of observations and conclusions, which are of great significance for future work, such as interpreting the intrinsic mechanisms of low-level CNN networks and developing new evaluation approaches for blind SR.

Related papers

• Coding schemes in neural networks learning classification tasks [52.22978725954347]
  We investigate fully-connected, wide neural networks learning classification tasks. We show that the networks acquire strong, data-dependent features. Surprisingly, the nature of the internal representations depends crucially on the neuronal nonlinearity.
  arXiv  Detail & Related papers  (2024-06-24T14:50:05Z)
• SeD: Semantic-Aware Discriminator for Image Super-Resolution [20.646975821512395]
  Generative Adversarial Networks (GANs) have been widely used to recover vivid textures in image super-resolution (SR) tasks. One discriminator is utilized to enable the SR network to learn the distribution of real-world high-quality images in an adversarial training manner. We propose the simple and effective Semantic-aware Discriminator ( SeD) SeD encourages the SR network to learn the fine-grained distributions by introducing the semantics of images as a condition.
  arXiv  Detail & Related papers  (2024-02-29T17:38:54Z)
• SAR Despeckling Using Overcomplete Convolutional Networks [53.99620005035804]
  despeckling is an important problem in remote sensing as speckle degrades SAR images. Recent studies show that convolutional neural networks(CNNs) outperform classical despeckling methods. This study employs an overcomplete CNN architecture to focus on learning low-level features by restricting the receptive field. We show that the proposed network improves despeckling performance compared to recent despeckling methods on synthetic and real SAR images.
  arXiv  Detail & Related papers  (2022-05-31T15:55:37Z)
• Textural-Structural Joint Learning for No-Reference Super-Resolution Image Quality Assessment [59.91741119995321]
  We develop a dual stream network to jointly explore the textural and structural information for quality prediction, dubbed TSNet. By mimicking the human vision system (HVS) that pays more attention to the significant areas of the image, we develop the spatial attention mechanism to make the visual-sensitive areas more distinguishable. Experimental results show the proposed TSNet predicts the visual quality more accurate than the state-of-the-art IQA methods, and demonstrates better consistency with the human's perspective.
  arXiv  Detail & Related papers  (2022-05-27T09:20:06Z)
• ADMM-DAD net: a deep unfolding network for analysis compressed sensing [20.88999913266683]
  We propose a new deep unfolding neural network based on the ADMM algorithm for analysis Compressed Sensing. The proposed network jointly learns a redundant analysis operator for sparsification and reconstructs the signal of interest.
  arXiv  Detail & Related papers  (2021-10-13T18:56:59Z)
• NetReAct: Interactive Learning for Network Summarization [60.18513812680714]
  We present NetReAct, a novel interactive network summarization algorithm which supports the visualization of networks induced by text corpora to perform sensemaking. We show how NetReAct is successful in generating high-quality summaries and visualizations that reveal hidden patterns better than other non-trivial baselines.
  arXiv  Detail & Related papers  (2020-12-22T03:56:26Z)
• Interpreting Super-Resolution Networks with Local Attribution Maps [24.221989130005085]
  Image super-resolution (SR) techniques have been developing rapidly, benefiting from the invention of deep networks and its successive breakthroughs. It is acknowledged that deep learning and deep neural networks are difficult to interpret. In this paper, we perform attribution analysis of SR networks, which aims at finding the input pixels that strongly influence the SR results.
  arXiv  Detail & Related papers  (2020-11-22T15:11:00Z)
• Learning Deep Interleaved Networks with Asymmetric Co-Attention for Image Restoration [65.11022516031463]
  We present a deep interleaved network (DIN) that learns how information at different states should be combined for high-quality (HQ) images reconstruction. In this paper, we propose asymmetric co-attention (AsyCA) which is attached at each interleaved node to model the feature dependencies. Our presented DIN can be trained end-to-end and applied to various image restoration tasks.
  arXiv  Detail & Related papers  (2020-10-29T15:32:00Z)
• SNoRe: Scalable Unsupervised Learning of Symbolic Node Representations [0.0]
  The proposed SNoRe algorithm is capable of learning symbolic, human-understandable representations of individual network nodes. SNoRe's interpretable features are suitable for direct explanation of individual predictions. The vectorized implementation of SNoRe scales to large networks, making it suitable for contemporary network learning and analysis tasks.
  arXiv  Detail & Related papers  (2020-09-08T08:13:21Z)
• Deep Interleaved Network for Image Super-Resolution With Asymmetric Co-Attention [11.654141322782074]
  We propose a deep interleaved network (DIN) to learn how information at different states should be combined for image SR. Our DIN follows a multi-branch pattern allowing multiple interconnected branches to interleave and fuse at different states. Besides, the asymmetric co-attention (AsyCA) is proposed and attacked to the interleaved nodes to adaptively emphasize informative features from different states.
  arXiv  Detail & Related papers  (2020-04-24T15:49:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.