SpArX: Sparse Argumentative Explanations for Neural Networks [Technical
Report]
- URL: http://arxiv.org/abs/2301.09559v3
- Date: Mon, 31 Jul 2023 09:13:32 GMT
- Title: SpArX: Sparse Argumentative Explanations for Neural Networks [Technical
Report]
- Authors: Hamed Ayoobi, Nico Potyka, Francesca Toni
- Abstract summary: We exploit relationships between multi-layer perceptrons (MLPs) and quantitative argumentation frameworks (QAFs) to create argumentative explanations for the mechanics of neural networks (NNs)
Our SpArX method first sparsifies the sparse while maintaining as much of the original structure as possible. It then translates, producing global and/or local explanations.
We demonstrate experimentally that SpArX can give more faithful explanations than existing approaches, while simultaneously providing deeper insights into the actual reasoning process of neural networks.
- Score: 14.787292425343527
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Neural networks (NNs) have various applications in AI, but explaining their
decisions remains challenging. Existing approaches often focus on explaining
how changing individual inputs affects NNs' outputs. However, an explanation
that is consistent with the input-output behaviour of an NN is not necessarily
faithful to the actual mechanics thereof. In this paper, we exploit
relationships between multi-layer perceptrons (MLPs) and quantitative
argumentation frameworks (QAFs) to create argumentative explanations for the
mechanics of MLPs. Our SpArX method first sparsifies the MLP while maintaining
as much of the original structure as possible. It then translates the sparse
MLP into an equivalent QAF to shed light on the underlying decision process of
the MLP, producing global and/or local explanations. We demonstrate
experimentally that SpArX can give more faithful explanations than existing
approaches, while simultaneously providing deeper insights into the actual
reasoning process of MLPs.
Related papers
- KAN or MLP: A Fairer Comparison [63.794304207664176]
This paper offers a fairer and more comprehensive comparison of KAN and models across various tasks.
We control the number of parameters and FLOPs to compare the performance of KAN and representation.
We find that KAN's issue is more severe than that of forgetting in a standard class-incremental continual learning setting.
arXiv Detail & Related papers (2024-07-23T17:43:35Z) - Verification and Refinement of Natural Language Explanations through LLM-Symbolic Theorem Proving [13.485604499678262]
This paper investigates the verification and refinement of natural language explanations through the integration of Large Language Models (LLMs) and Theorem Provers (TPs)
We present a neuro-symbolic framework, named Explanation-Refiner, that augments a TP with LLMs to generate and formalise explanatory sentences.
In turn, the TP is employed to provide formal guarantees on the logical validity of the explanations and to generate feedback for subsequent improvements.
arXiv Detail & Related papers (2024-05-02T15:20:01Z) - Interpreting Key Mechanisms of Factual Recall in Transformer-Based Language Models [68.83330172211315]
We study mechanisms employed by Transformer-based language models (LLMs) for factual recall tasks.
We propose a novel analytic method aimed at decomposing the outputs of the outputs into components understandable by humans.
We mitigate this suppression by leveraging our interpretation to improve factual recall confidence.
arXiv Detail & Related papers (2024-03-28T15:54:59Z) - FaithLM: Towards Faithful Explanations for Large Language Models [67.29893340289779]
Large Language Models (LLMs) have become proficient in addressing complex tasks by leveraging their internal knowledge and reasoning capabilities.
The black-box nature of these models complicates the task of explaining their decision-making processes.
We introduce FaithLM to explain the decision of LLMs with natural language (NL) explanations.
arXiv Detail & Related papers (2024-02-07T09:09:14Z) - MLPs Compass: What is learned when MLPs are combined with PLMs? [20.003022732050994]
Multilayer-Perceptrons (MLPs) modules achieving robust structural capture capabilities, even outperforming Graph Neural Networks (GNNs)
This paper aims to quantify whether simples can further enhance the already potent ability of PLMs to capture linguistic information.
arXiv Detail & Related papers (2024-01-03T11:06:01Z) - NTK-approximating MLP Fusion for Efficient Language Model Fine-tuning [40.994306592119266]
Fine-tuning a pre-trained language model (PLM) emerges as the predominant strategy in many natural language processing applications.
Some general approaches (e.g. quantization and distillation) have been widely studied to reduce the compute/memory of PLM fine-tuning.
We propose to coin a lightweight PLM through NTK-approximating modules in fusion.
arXiv Detail & Related papers (2023-07-18T03:12:51Z) - SA-MLP: Distilling Graph Knowledge from GNNs into Structure-Aware MLP [46.52398427166938]
One promising inference acceleration direction is to distill the GNNs into message-passing-free student multi-layer perceptrons.
We introduce a novel structure-mixing knowledge strategy to enhance the learning ability of students for structure information.
Our SA-MLP can consistently outperform the teacher GNNs, while maintaining faster inference assitance.
arXiv Detail & Related papers (2022-10-18T05:55:36Z) - Transformer Vs. MLP-Mixer Exponential Expressive Gap For NLP Problems [8.486025595883117]
We analyze the expressive power of mlp-based architectures in modeling dependencies between multiple inputs simultaneously.
We show an exponential gap between the attention and the mlp-based mechanisms.
Our results suggest a theoretical explanation for the mlp inability to compete with attention-based mechanisms in NLP problems.
arXiv Detail & Related papers (2022-08-17T09:59:22Z) - How Neural Networks Extrapolate: From Feedforward to Graph Neural
Networks [80.55378250013496]
We study how neural networks trained by gradient descent extrapolate what they learn outside the support of the training distribution.
Graph Neural Networks (GNNs) have shown some success in more complex tasks.
arXiv Detail & Related papers (2020-09-24T17:48:59Z) - MPLP: Learning a Message Passing Learning Protocol [63.948465205530916]
We present a novel method for learning the weights of an artificial neural network - a Message Passing Learning Protocol (MPLP)
We abstract every operations occurring in ANNs as independent agents.
Each agent is responsible for ingesting incoming multidimensional messages from other agents, updating its internal state, and generating multidimensional messages to be passed on to neighbouring agents.
arXiv Detail & Related papers (2020-07-02T09:03:14Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.