Interpreting the Residual Stream of ResNet18

• URL: http://arxiv.org/abs/2407.05340v1
• Date: Sun, 7 Jul 2024 12:13:03 GMT
• Title: Interpreting the Residual Stream of ResNet18
• Authors: André Longon,
• Abstract summary: This work investigates ResNet18 with a particular focus on its residual stream, an architectural mechanism which InceptionV1 lacks. We show that many residual stream channels compute scale invariant representations through a mixture of the input's smaller-scale feature with the block's larger-scale feature.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A mechanistic understanding of the computations learned by deep neural networks (DNNs) is far from complete. In the domain of visual object recognition, prior research has illuminated inner workings of InceptionV1, but DNNs with different architectures have remained largely unexplored. This work investigates ResNet18 with a particular focus on its residual stream, an architectural mechanism which InceptionV1 lacks. We observe that for a given block, channel features of the stream are updated along a spectrum: either the input feature skips to the output, the block feature overwrites the output, or the output is some mixture between the input and block features. Furthermore, we show that many residual stream channels compute scale invariant representations through a mixture of the input's smaller-scale feature with the block's larger-scale feature. This not only mounts evidence for the universality of scale equivariance, but also presents how the residual stream further implements scale invariance. Collectively, our results begin an interpretation of the residual stream in visual object recognition, finding it to be a flexible feature manager and a medium to build scale invariant representations.

Related papers

• Naturally Computed Scale Invariance in the Residual Stream of ResNet18 [0.0]
  This work investigates ResNet18 with a particular focus on its residual stream, an architectural component which InceptionV1 lacks. We observe that many convolutional channels in intermediate blocks exhibit scale invariant properties, computed by the element-wise residual summation of scale equivariant representations. Through subsequent ablation experiments, we attempt to causally link these neural properties with scale-robust object recognition behavior.
  arXiv  Detail & Related papers  (2025-04-22T21:54:37Z)
• Accurate and lightweight dehazing via multi-receptive-field non-local network and novel contrastive regularization [9.90146712189936]
  This paper presents a multi-receptive-field non-local network (MRFNLN) for image dehazing. It is designed as a multi-stream feature attention block (MSFAB) and cross non-local block (CNLB) It outperforms recent state-of-the-art dehazing methods with less than 1.5 Million parameters.
  arXiv  Detail & Related papers  (2023-09-28T14:59:16Z)
• Effective and Interpretable Information Aggregation with Capacity Networks [3.4012007729454807]
  Capacity networks generate multiple interpretable intermediate results which can be aggregated in a semantically meaningful space. Our experiments show that implementing this simple inductive bias leads to improvements over different encoder-decoder architectures.
  arXiv  Detail & Related papers  (2022-07-25T09:45:16Z)
• Towards Efficient Scene Understanding via Squeeze Reasoning [71.1139549949694]
  We propose a novel framework called Squeeze Reasoning. Instead of propagating information on the spatial map, we first learn to squeeze the input feature into a channel-wise global vector. We show that our approach can be modularized as an end-to-end trained block and can be easily plugged into existing networks.
  arXiv  Detail & Related papers  (2020-11-06T12:17:01Z)
• Feature Flow: In-network Feature Flow Estimation for Video Object Detection [56.80974623192569]
  Optical flow is widely used in computer vision tasks to provide pixel-level motion information. A common approach is to:forward optical flow to a neural network and fine-tune this network on the task dataset. We propose a novel network (IFF-Net) with an textbfIn-network textbfFeature textbfFlow estimation module for video object detection.
  arXiv  Detail & Related papers  (2020-09-21T07:55:50Z)
• Volumetric Transformer Networks [88.85542905676712]
  We introduce a learnable module, the volumetric transformer network (VTN) VTN predicts channel-wise warping fields so as to reconfigure intermediate CNN features spatially and channel-wisely. Our experiments show that VTN consistently boosts the features' representation power and consequently the networks' accuracy on fine-grained image recognition and instance-level image retrieval.
  arXiv  Detail & Related papers  (2020-07-18T14:00:12Z)
• Multi-scale Interactive Network for Salient Object Detection [91.43066633305662]
  We propose the aggregate interaction modules to integrate the features from adjacent levels. To obtain more efficient multi-scale features, the self-interaction modules are embedded in each decoder unit. Experimental results on five benchmark datasets demonstrate that the proposed method without any post-processing performs favorably against 23 state-of-the-art approaches.
  arXiv  Detail & Related papers  (2020-07-17T15:41:37Z)
• Adaptive feature recombination and recalibration for semantic segmentation with Fully Convolutional Networks [57.64866581615309]
  We propose recombination of features and a spatially adaptive recalibration block that is adapted for semantic segmentation with Fully Convolutional Networks. Results indicate that Recombination and Recalibration improve the results of a competitive baseline, and generalize across three different problems.
  arXiv  Detail & Related papers  (2020-06-19T15:45:03Z)
• ResNeSt: Split-Attention Networks [86.25490825631763]
  We present a modularized architecture, which applies the channel-wise attention on different network branches to leverage their success in capturing cross-feature interactions and learning diverse representations. Our model, named ResNeSt, outperforms EfficientNet in accuracy and latency trade-off on image classification.
  arXiv  Detail & Related papers  (2020-04-19T20:40:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.