Causality-aware counterfactual confounding adjustment for feature representations learned by deep models

• URL: http://arxiv.org/abs/2004.09466v4
• Date: Fri, 20 Nov 2020 03:25:16 GMT
• Title: Causality-aware counterfactual confounding adjustment for feature representations learned by deep models
• Authors: Elias Chaibub Neto
• Abstract summary: Causal modeling has been recognized as a potential solution to many challenging problems in machine learning (ML) We describe how a recently proposed counterfactual approach can still be used to deconfound the feature representations learned by deep neural network (DNN) models.
• Score: 14.554818659491644
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Causal modeling has been recognized as a potential solution to many challenging problems in machine learning (ML). Here, we describe how a recently proposed counterfactual approach developed to deconfound linear structural causal models can still be used to deconfound the feature representations learned by deep neural network (DNN) models. The key insight is that by training an accurate DNN using softmax activation at the classification layer, and then adopting the representation learned by the last layer prior to the output layer as our features, we have that, by construction, the learned features will fit well a (multi-class) logistic regression model, and will be linearly associated with the labels. As a consequence, deconfounding approaches based on simple linear models can be used to deconfound the feature representations learned by DNNs. We validate the proposed methodology using colored versions of the MNIST dataset. Our results illustrate how the approach can effectively combat confounding and improve model stability in the context of dataset shifts generated by selection biases.

Related papers

• Deep learning for model correction of dynamical systems with data scarcity [0.0]
  We present a deep learning framework for correcting existing dynamical system models utilizing only a scarce high-fidelity data set. We focus on the case when the amount of high-fidelity data is so small that most of the existing data driven modeling methods cannot be applied.
  arXiv  Detail & Related papers  (2024-10-23T14:33:11Z)
• Neural Additive Models for Location Scale and Shape: A Framework for Interpretable Neural Regression Beyond the Mean [1.0923877073891446]
  Deep neural networks (DNNs) have proven to be highly effective in a variety of tasks. Despite this success, the inner workings of DNNs are often not transparent. This lack of interpretability has led to increased research on inherently interpretable neural networks.
  arXiv  Detail & Related papers  (2023-01-27T17:06:13Z)
• Dynamically-Scaled Deep Canonical Correlation Analysis [77.34726150561087]
  Canonical Correlation Analysis (CCA) is a method for feature extraction of two views by finding maximally correlated linear projections of them. We introduce a novel dynamic scaling method for training an input-dependent canonical correlation model.
  arXiv  Detail & Related papers  (2022-03-23T12:52:49Z)
• Sparse Flows: Pruning Continuous-depth Models [107.98191032466544]
  We show that pruning improves generalization for neural ODEs in generative modeling. We also show that pruning finds minimal and efficient neural ODE representations with up to 98% less parameters compared to the original network, without loss of accuracy.
  arXiv  Detail & Related papers  (2021-06-24T01:40:17Z)
• Gone Fishing: Neural Active Learning with Fisher Embeddings [55.08537975896764]
  There is an increasing need for active learning algorithms that are compatible with deep neural networks. This article introduces BAIT, a practical representation of tractable, and high-performing active learning algorithm for neural networks.
  arXiv  Detail & Related papers  (2021-06-17T17:26:31Z)
• Self-interpretable Convolutional Neural Networks for Text Classification [5.55878488884108]
  This paper develops an approach for interpreting convolutional neural networks for text classification problems by exploiting the local-linear models inherent in ReLU-DNNs. We show that our proposed technique produce parsimonious models that are self-interpretable and have comparable performance with respect to a more complex CNN model.
  arXiv  Detail & Related papers  (2021-05-18T15:19:59Z)
• Rank-R FNN: A Tensor-Based Learning Model for High-Order Data Classification [69.26747803963907]
  Rank-R Feedforward Neural Network (FNN) is a tensor-based nonlinear learning model that imposes Canonical/Polyadic decomposition on its parameters. First, it handles inputs as multilinear arrays, bypassing the need for vectorization, and can thus fully exploit the structural information along every data dimension. We establish the universal approximation and learnability properties of Rank-R FNN, and we validate its performance on real-world hyperspectral datasets.
  arXiv  Detail & Related papers  (2021-04-11T16:37:32Z)
• Explaining and Improving Model Behavior with k Nearest Neighbor Representations [107.24850861390196]
  We propose using k nearest neighbor representations to identify training examples responsible for a model's predictions. We show that kNN representations are effective at uncovering learned spurious associations. Our results indicate that the kNN approach makes the finetuned model more robust to adversarial inputs.
  arXiv  Detail & Related papers  (2020-10-18T16:55:25Z)
• Belief Propagation Reloaded: Learning BP-Layers for Labeling Problems [83.98774574197613]
  We take one of the simplest inference methods, a truncated max-product Belief propagation, and add what is necessary to make it a proper component of a deep learning model. This BP-Layer can be used as the final or an intermediate block in convolutional neural networks (CNNs) The model is applicable to a range of dense prediction problems, is well-trainable and provides parameter-efficient and robust solutions in stereo, optical flow and semantic segmentation.
  arXiv  Detail & Related papers  (2020-03-13T13:11:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.