Much Easier Said Than Done: Falsifying the Causal Relevance of Linear Decoding Methods

• URL: http://arxiv.org/abs/2211.04367v1
• Date: Tue, 8 Nov 2022 16:43:02 GMT
• Title: Much Easier Said Than Done: Falsifying the Causal Relevance of Linear Decoding Methods
• Authors: Lucas Hayne, Abhijit Suresh, Hunar Jain, Rahul Kumar, R. McKell Carter
• Abstract summary: Linear classifier probes identify highly selective units as the most important for network function. In spite of the absence of ablation effects for selective neurons, linear decoding methods can be effectively used to interpret network function. More specifically, we find that an interaction between selectivity and the average activity of the unit better predicts ablation performance deficits for groups of units in AlexNet, VGG16, MobileNetV2, and ResNet101.
• Score: 1.3999481573773074
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Linear classifier probes are frequently utilized to better understand how neural networks function. Researchers have approached the problem of determining unit importance in neural networks by probing their learned, internal representations. Linear classifier probes identify highly selective units as the most important for network function. Whether or not a network actually relies on high selectivity units can be tested by removing them from the network using ablation. Surprisingly, when highly selective units are ablated they only produce small performance deficits, and even then only in some cases. In spite of the absence of ablation effects for selective neurons, linear decoding methods can be effectively used to interpret network function, leaving their effectiveness a mystery. To falsify the exclusive role of selectivity in network function and resolve this contradiction, we systematically ablate groups of units in subregions of activation space. Here, we find a weak relationship between neurons identified by probes and those identified by ablation. More specifically, we find that an interaction between selectivity and the average activity of the unit better predicts ablation performance deficits for groups of units in AlexNet, VGG16, MobileNetV2, and ResNet101. Linear decoders are likely somewhat effective because they overlap with those units that are causally important for network function. Interpretability methods could be improved by focusing on causally important units.

Related papers

• Coding schemes in neural networks learning classification tasks [52.22978725954347]
  We investigate fully-connected, wide neural networks learning classification tasks. We show that the networks acquire strong, data-dependent features. Surprisingly, the nature of the internal representations depends crucially on the neuronal nonlinearity.
  arXiv  Detail & Related papers  (2024-06-24T14:50:05Z)
• Effective Subset Selection Through The Lens of Neural Network Pruning [31.43307762723943]
  It is important to select the data to be annotated wisely, which is known as the subset selection problem. We investigate the relationship between subset selection and neural network pruning, which is more widely studied. We propose utilizing the norm criterion of neural network features to improve subset selection methods.
  arXiv  Detail & Related papers  (2024-06-03T08:12:32Z)
• An Adaptive Contrastive Learning Model for Spike Sorting [12.043679000694258]
  In neuroscience research, it is important to separate out the activity of individual neurons. With the development of large-scale silicon technology, artificially interpreting and labeling spikes is becoming increasingly impractical. We propose a novel modeling framework that learns representations from spikes through contrastive learning.
  arXiv  Detail & Related papers  (2022-05-24T09:18:46Z)
• Wide and Deep Neural Networks Achieve Optimality for Classification [23.738242876364865]
  We identify and construct an explicit set of neural network classifiers that achieve optimality. In particular, we provide explicit activation functions that can be used to construct networks that achieve optimality. Our results highlight the benefit of using deep networks for classification tasks, in contrast to regression tasks, where excessive depth is harmful.
  arXiv  Detail & Related papers  (2022-04-29T14:27:42Z)
• And/or trade-off in artificial neurons: impact on adversarial robustness [91.3755431537592]
  Presence of sufficient number of OR-like neurons in a network can lead to classification brittleness and increased vulnerability to adversarial attacks. We define AND-like neurons and propose measures to increase their proportion in the network. Experimental results on the MNIST dataset suggest that our approach holds promise as a direction for further exploration.
  arXiv  Detail & Related papers  (2021-02-15T08:19:05Z)
• The Connection Between Approximation, Depth Separation and Learnability in Neural Networks [70.55686685872008]
  We study the connection between learnability and approximation capacity. We show that learnability with deep networks of a target function depends on the ability of simpler classes to approximate the target.
  arXiv  Detail & Related papers  (2021-01-31T11:32:30Z)
• Learning Connectivity of Neural Networks from a Topological Perspective [80.35103711638548]
  We propose a topological perspective to represent a network into a complete graph for analysis. By assigning learnable parameters to the edges which reflect the magnitude of connections, the learning process can be performed in a differentiable manner. This learning process is compatible with existing networks and owns adaptability to larger search spaces and different tasks.
  arXiv  Detail & Related papers  (2020-08-19T04:53:31Z)
• Consistent feature selection for neural networks via Adaptive Group Lasso [3.42658286826597]
  We propose and establish a theoretical guarantee for the use of the adaptive group for selecting important features of neural networks. Specifically, we show that our feature selection method is consistent for single-output feed-forward neural networks with one hidden layer and hyperbolic tangent activation function.
  arXiv  Detail & Related papers  (2020-05-30T18:50:56Z)
• Towards Efficient Processing and Learning with Spikes: New Approaches for Multi-Spike Learning [59.249322621035056]
  We propose two new multi-spike learning rules which demonstrate better performance over other baselines on various tasks. In the feature detection task, we re-examine the ability of unsupervised STDP with its limitations being presented. Our proposed learning rules can reliably solve the task over a wide range of conditions without specific constraints being applied.
  arXiv  Detail & Related papers  (2020-05-02T06:41:20Z)
• BiDet: An Efficient Binarized Object Detector [96.19708396510894]
  We propose a binarized neural network learning method called BiDet for efficient object detection. Our BiDet fully utilizes the representational capacity of the binary neural networks for object detection by redundancy removal. Our method outperforms the state-of-the-art binary neural networks by a sizable margin.
  arXiv  Detail & Related papers  (2020-03-09T08:16:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.