Higher-order Neural Additive Models: An Interpretable Machine Learning
Model with Feature Interactions
- URL: http://arxiv.org/abs/2209.15409v1
- Date: Fri, 30 Sep 2022 12:12:30 GMT
- Title: Higher-order Neural Additive Models: An Interpretable Machine Learning
Model with Feature Interactions
- Authors: Minkyu Kim, Hyun-Soo Choi, Jinho Kim
- Abstract summary: Black-box models, such as deep neural networks, exhibit superior predictive performances, but understanding their behavior is notoriously difficult.
Recently proposed neural additive models (NAM) have achieved state-of-the-art interpretable machine learning.
We propose a novel interpretable machine learning method called higher-order neural additive models (HONAM) and a feature interaction method for high interpretability.
- Score: 2.127049691404299
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Black-box models, such as deep neural networks, exhibit superior predictive
performances, but understanding their behavior is notoriously difficult. Many
explainable artificial intelligence methods have been proposed to reveal the
decision-making processes of black box models. However, their applications in
high-stakes domains remain limited. Recently proposed neural additive models
(NAM) have achieved state-of-the-art interpretable machine learning. NAM can
provide straightforward interpretations with slight performance sacrifices
compared with multi-layer perceptron. However, NAM can only model
1$^{\text{st}}$-order feature interactions; thus, it cannot capture the
co-relationships between input features. To overcome this problem, we propose a
novel interpretable machine learning method called higher-order neural additive
models (HONAM) and a feature interaction method for high interpretability.
HONAM can model arbitrary orders of feature interactions. Therefore, it can
provide the high predictive performance and interpretability that high-stakes
domains need. In addition, we propose a novel hidden unit to effectively learn
sharp-shape functions. We conducted experiments using various real-world
datasets to examine the effectiveness of HONAM. Furthermore, we demonstrate
that HONAM can achieve fair AI with a slight performance sacrifice. The source
code for HONAM is publicly available.
Related papers
- Interpretable Language Modeling via Induction-head Ngram Models [74.26720927767398]
We propose Induction-head ngram models (Induction-Gram) to bolster modern ngram models with a hand-engineered "induction head"
This induction head uses a custom neural similarity metric to efficiently search the model's input context for potential next-word completions.
Experiments show that this simple method significantly improves next-word prediction over baseline interpretable models.
arXiv Detail & Related papers (2024-10-31T12:33:26Z) - Explainable Automatic Grading with Neural Additive Models [0.0]
Current state-of-the-art ASAG models are large neural networks (NN) often described as "black box"
We experiment with a type of model called a Neural Additive Model that combines the performance of a NN with the explainability of an additive model.
arXiv Detail & Related papers (2024-05-01T12:56:14Z) - Efficient Adaptive Human-Object Interaction Detection with
Concept-guided Memory [64.11870454160614]
We propose an efficient Adaptive HOI Detector with Concept-guided Memory (ADA-CM)
ADA-CM has two operating modes. The first mode makes it tunable without learning new parameters in a training-free paradigm.
Our proposed method achieves competitive results with state-of-the-art on the HICO-DET and V-COCO datasets with much less training time.
arXiv Detail & Related papers (2023-09-07T13:10:06Z) - Value function estimation using conditional diffusion models for control [62.27184818047923]
We propose a simple algorithm called Diffused Value Function (DVF)
It learns a joint multi-step model of the environment-robot interaction dynamics using a diffusion model.
We show how DVF can be used to efficiently capture the state visitation measure for multiple controllers.
arXiv Detail & Related papers (2023-06-09T18:40:55Z) - Structural Neural Additive Models: Enhanced Interpretable Machine
Learning [0.0]
In recent years, the field has seen a push towards interpretable neural networks, such as the visually interpretable Neural Additive Models (NAMs)
We propose a further step into the direction of intelligibility beyond the mere visualization of feature effects and propose Structural Neural Additive Models (SNAMs)
A modeling framework that combines classical and clearly interpretable statistical methods with the predictive power of neural applications.
arXiv Detail & Related papers (2023-02-18T09:52:30Z) - Neural Additive Models for Location Scale and Shape: A Framework for
Interpretable Neural Regression Beyond the Mean [1.0923877073891446]
Deep neural networks (DNNs) have proven to be highly effective in a variety of tasks.
Despite this success, the inner workings of DNNs are often not transparent.
This lack of interpretability has led to increased research on inherently interpretable neural networks.
arXiv Detail & Related papers (2023-01-27T17:06:13Z) - Neural Additive Models for Nowcasting [1.8275108630751844]
We propose neural additive models (NAMs) to provide explanatory power for neural network predictions.
We show that the proposed NAM-NC successfully explains each input value's importance for multiple variables and time steps.
We also examine parameter-sharing networks using NAM-NC to decrease their complexity, and NAM-MC's hard-tied feature net extracted explanations with good performance.
arXiv Detail & Related papers (2022-05-20T08:25:18Z) - Dynamic Inference with Neural Interpreters [72.90231306252007]
We present Neural Interpreters, an architecture that factorizes inference in a self-attention network as a system of modules.
inputs to the model are routed through a sequence of functions in a way that is end-to-end learned.
We show that Neural Interpreters perform on par with the vision transformer using fewer parameters, while being transferrable to a new task in a sample efficient manner.
arXiv Detail & Related papers (2021-10-12T23:22:45Z) - Gone Fishing: Neural Active Learning with Fisher Embeddings [55.08537975896764]
There is an increasing need for active learning algorithms that are compatible with deep neural networks.
This article introduces BAIT, a practical representation of tractable, and high-performing active learning algorithm for neural networks.
arXiv Detail & Related papers (2021-06-17T17:26:31Z) - Neural Additive Models: Interpretable Machine Learning with Neural Nets [77.66871378302774]
Deep neural networks (DNNs) are powerful black-box predictors that have achieved impressive performance on a wide variety of tasks.
We propose Neural Additive Models (NAMs) which combine some of the expressivity of DNNs with the inherent intelligibility of generalized additive models.
NAMs learn a linear combination of neural networks that each attend to a single input feature.
arXiv Detail & Related papers (2020-04-29T01:28:32Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.