Contrastive Reasoning in Neural Networks

• URL: http://arxiv.org/abs/2103.12329v1
• Date: Tue, 23 Mar 2021 05:54:36 GMT
• Title: Contrastive Reasoning in Neural Networks
• Authors: Mohit Prabhushankar and Ghassan AlRegib
• Abstract summary: Inference built on features that identify causal class dependencies is termed as feed-forward inference. In this paper, we formalize the structure of contrastive reasoning and propose a methodology to extract a neural network's notion of contrast. We demonstrate the value of contrastively recognizing images under distortions by reporting an improvement of 3.47%, 2.56%, and 5.48% in average accuracy.
• Score: 26.65337569468343
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Neural networks represent data as projections on trained weights in a high dimensional manifold. The trained weights act as a knowledge base consisting of causal class dependencies. Inference built on features that identify these dependencies is termed as feed-forward inference. Such inference mechanisms are justified based on classical cause-to-effect inductive reasoning models. Inductive reasoning based feed-forward inference is widely used due to its mathematical simplicity and operational ease. Nevertheless, feed-forward models do not generalize well to untrained situations. To alleviate this generalization challenge, we propose using an effect-to-cause inference model that reasons abductively. Here, the features represent the change from existing weight dependencies given a certain effect. We term this change as contrast and the ensuing reasoning mechanism as contrastive reasoning. In this paper, we formalize the structure of contrastive reasoning and propose a methodology to extract a neural network's notion of contrast. We demonstrate the value of contrastive reasoning in two stages of a neural network's reasoning pipeline : in inferring and visually explaining decisions for the application of object recognition. We illustrate the value of contrastively recognizing images under distortions by reporting an improvement of 3.47%, 2.56%, and 5.48% in average accuracy under the proposed contrastive framework on CIFAR-10C, noisy STL-10, and VisDA datasets respectively.

Related papers

• Improving Network Interpretability via Explanation Consistency Evaluation [56.14036428778861]
  We propose a framework that acquires more explainable activation heatmaps and simultaneously increase the model performance. Specifically, our framework introduces a new metric, i.e., explanation consistency, to reweight the training samples adaptively in model learning. Our framework then promotes the model learning by paying closer attention to those training samples with a high difference in explanations.
  arXiv  Detail & Related papers  (2024-08-08T17:20:08Z)
• Empirical influence functions to understand the logic of fine-tuning [1.9116784879310031]
  We use empirical influence measured using fine-tuning to demonstrate how individual training samples affect outputs. We show that these desiderata are violated for both for simple convolutional networks and for a modern LLM. Our results suggest that popular models cannot generalize or perform logic in the way they appear.
  arXiv  Detail & Related papers  (2024-06-01T17:31:06Z)
• Causal Fairness-Guided Dataset Reweighting using Neural Networks [25.492273448917278]
  We construct a reweighting scheme of datasets to address causal fairness. Our approach aims at mitigating bias by considering the causal relationships among variables. We show that our method can achieve causal fairness on the data while remaining close to the original data for downstream tasks.
  arXiv  Detail & Related papers  (2023-11-17T13:31:19Z)
• Interpretable Imitation Learning with Dynamic Causal Relations [65.18456572421702]
  We propose to expose captured knowledge in the form of a directed acyclic causal graph. We also design this causal discovery process to be state-dependent, enabling it to model the dynamics in latent causal graphs. The proposed framework is composed of three parts: a dynamic causal discovery module, a causality encoding module, and a prediction module, and is trained in an end-to-end manner.
  arXiv  Detail & Related papers  (2023-09-30T20:59:42Z)
• Advancing Counterfactual Inference through Nonlinear Quantile Regression [77.28323341329461]
  We propose a framework for efficient and effective counterfactual inference implemented with neural networks. The proposed approach enhances the capacity to generalize estimated counterfactual outcomes to unseen data. Empirical results conducted on multiple datasets offer compelling support for our theoretical assertions.
  arXiv  Detail & Related papers  (2023-06-09T08:30:51Z)
• Causal Analysis for Robust Interpretability of Neural Networks [0.2519906683279152]
  We develop a robust interventional-based method to capture cause-effect mechanisms in pre-trained neural networks. We apply our method to vision models trained on classification tasks.
  arXiv  Detail & Related papers  (2023-05-15T18:37:24Z)
• Neural Causal Models for Counterfactual Identification and Estimation [62.30444687707919]
  We study the evaluation of counterfactual statements through neural models. First, we show that neural causal models (NCMs) are expressive enough. Second, we develop an algorithm for simultaneously identifying and estimating counterfactual distributions.
  arXiv  Detail & Related papers  (2022-09-30T18:29:09Z)
• How Tempering Fixes Data Augmentation in Bayesian Neural Networks [22.188535244056016]
  We show that tempering implicitly reduces the misspecification arising from modeling augmentations as i.i.d. data. The temperature mimics the role of the effective sample size, reflecting the gain in information provided by the augmentations.
  arXiv  Detail & Related papers  (2022-05-27T11:06:56Z)
• The Causal Neural Connection: Expressiveness, Learnability, and Inference [125.57815987218756]
  An object called structural causal model (SCM) represents a collection of mechanisms and sources of random variation of the system under investigation. In this paper, we show that the causal hierarchy theorem (Thm. 1, Bareinboim et al., 2020) still holds for neural models. We introduce a special type of SCM called a neural causal model (NCM), and formalize a new type of inductive bias to encode structural constraints necessary for performing causal inferences.
  arXiv  Detail & Related papers  (2021-07-02T01:55:18Z)
• Explain by Evidence: An Explainable Memory-based Neural Network for Question Answering [41.73026155036886]
  This paper proposes an explainable, evidence-based memory network architecture. It learns to summarize the dataset and extract supporting evidences to make its decision. Our model achieves state-of-the-art performance on two popular question answering datasets.
  arXiv  Detail & Related papers  (2020-11-05T21:18:21Z)
• Structural Causal Models Are (Solvable by) Credal Networks [70.45873402967297]
  Causal inferences can be obtained by standard algorithms for the updating of credal nets. This contribution should be regarded as a systematic approach to represent structural causal models by credal networks. Experiments show that approximate algorithms for credal networks can immediately be used to do causal inference in real-size problems.
  arXiv  Detail & Related papers  (2020-08-02T11:19:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.