Neural Additive Models: Interpretable Machine Learning with Neural Nets
- URL: http://arxiv.org/abs/2004.13912v2
- Date: Sun, 24 Oct 2021 19:24:59 GMT
- Title: Neural Additive Models: Interpretable Machine Learning with Neural Nets
- Authors: Rishabh Agarwal, Levi Melnick, Nicholas Frosst, Xuezhou Zhang, Ben
Lengerich, Rich Caruana, Geoffrey Hinton
- Abstract summary: Deep neural networks (DNNs) are powerful black-box predictors that have achieved impressive performance on a wide variety of tasks.
We propose Neural Additive Models (NAMs) which combine some of the expressivity of DNNs with the inherent intelligibility of generalized additive models.
NAMs learn a linear combination of neural networks that each attend to a single input feature.
- Score: 77.66871378302774
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Deep neural networks (DNNs) are powerful black-box predictors that have
achieved impressive performance on a wide variety of tasks. However, their
accuracy comes at the cost of intelligibility: it is usually unclear how they
make their decisions. This hinders their applicability to high stakes
decision-making domains such as healthcare. We propose Neural Additive Models
(NAMs) which combine some of the expressivity of DNNs with the inherent
intelligibility of generalized additive models. NAMs learn a linear combination
of neural networks that each attend to a single input feature. These networks
are trained jointly and can learn arbitrarily complex relationships between
their input feature and the output. Our experiments on regression and
classification datasets show that NAMs are more accurate than widely used
intelligible models such as logistic regression and shallow decision trees.
They perform similarly to existing state-of-the-art generalized additive models
in accuracy, but are more flexible because they are based on neural nets
instead of boosted trees. To demonstrate this, we show how NAMs can be used for
multitask learning on synthetic data and on the COMPAS recidivism data due to
their composability, and demonstrate that the differentiability of NAMs allows
them to train more complex interpretable models for COVID-19.
Related papers
- How neural networks learn to classify chaotic time series [77.34726150561087]
We study the inner workings of neural networks trained to classify regular-versus-chaotic time series.
We find that the relation between input periodicity and activation periodicity is key for the performance of LKCNN models.
arXiv Detail & Related papers (2023-06-04T08:53:27Z) - Transferability of coVariance Neural Networks and Application to
Interpretable Brain Age Prediction using Anatomical Features [119.45320143101381]
Graph convolutional networks (GCN) leverage topology-driven graph convolutional operations to combine information across the graph for inference tasks.
We have studied GCNs with covariance matrices as graphs in the form of coVariance neural networks (VNNs)
VNNs inherit the scale-free data processing architecture from GCNs and here, we show that VNNs exhibit transferability of performance over datasets whose covariance matrices converge to a limit object.
arXiv Detail & Related papers (2023-05-02T22:15:54Z) - Structural Neural Additive Models: Enhanced Interpretable Machine
Learning [0.0]
In recent years, the field has seen a push towards interpretable neural networks, such as the visually interpretable Neural Additive Models (NAMs)
We propose a further step into the direction of intelligibility beyond the mere visualization of feature effects and propose Structural Neural Additive Models (SNAMs)
A modeling framework that combines classical and clearly interpretable statistical methods with the predictive power of neural applications.
arXiv Detail & Related papers (2023-02-18T09:52:30Z) - Neural Additive Models for Location Scale and Shape: A Framework for
Interpretable Neural Regression Beyond the Mean [1.0923877073891446]
Deep neural networks (DNNs) have proven to be highly effective in a variety of tasks.
Despite this success, the inner workings of DNNs are often not transparent.
This lack of interpretability has led to increased research on inherently interpretable neural networks.
arXiv Detail & Related papers (2023-01-27T17:06:13Z) - Higher-order Neural Additive Models: An Interpretable Machine Learning
Model with Feature Interactions [2.127049691404299]
Black-box models, such as deep neural networks, exhibit superior predictive performances, but understanding their behavior is notoriously difficult.
Recently proposed neural additive models (NAM) have achieved state-of-the-art interpretable machine learning.
We propose a novel interpretable machine learning method called higher-order neural additive models (HONAM) and a feature interaction method for high interpretability.
arXiv Detail & Related papers (2022-09-30T12:12:30Z) - Simple and complex spiking neurons: perspectives and analysis in a
simple STDP scenario [0.7829352305480283]
Spiking neural networks (SNNs) are inspired by biology and neuroscience to create fast and efficient learning systems.
This work considers various neuron models in the literature and then selects computational neuron models that are single-variable, efficient, and display different types of complexities.
We make a comparative study of three simple I&F neuron models, namely the LIF, the Quadratic I&F (QIF) and the Exponential I&F (EIF), to understand whether the use of more complex models increases the performance of the system.
arXiv Detail & Related papers (2022-06-28T10:01:51Z) - Neural Additive Models for Nowcasting [1.8275108630751844]
We propose neural additive models (NAMs) to provide explanatory power for neural network predictions.
We show that the proposed NAM-NC successfully explains each input value's importance for multiple variables and time steps.
We also examine parameter-sharing networks using NAM-NC to decrease their complexity, and NAM-MC's hard-tied feature net extracted explanations with good performance.
arXiv Detail & Related papers (2022-05-20T08:25:18Z) - EINNs: Epidemiologically-Informed Neural Networks [75.34199997857341]
We introduce a new class of physics-informed neural networks-EINN-crafted for epidemic forecasting.
We investigate how to leverage both the theoretical flexibility provided by mechanistic models as well as the data-driven expressability afforded by AI models.
arXiv Detail & Related papers (2022-02-21T18:59:03Z) - Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs.
By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
arXiv Detail & Related papers (2022-02-01T17:11:13Z) - Dynamic Inference with Neural Interpreters [72.90231306252007]
We present Neural Interpreters, an architecture that factorizes inference in a self-attention network as a system of modules.
inputs to the model are routed through a sequence of functions in a way that is end-to-end learned.
We show that Neural Interpreters perform on par with the vision transformer using fewer parameters, while being transferrable to a new task in a sample efficient manner.
arXiv Detail & Related papers (2021-10-12T23:22:45Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.