Neural Additive Experts: Context-Gated Experts for Controllable Model Additivity

• URL: http://arxiv.org/abs/2602.10585v1
• Date: Wed, 11 Feb 2026 07:19:25 GMT
• Title: Neural Additive Experts: Context-Gated Experts for Controllable Model Additivity
• Authors: Guangzhi Xiong, Sanchit Sinha, Aidong Zhang,
• Abstract summary: We propose a novel framework that seamlessly balances interpretability and accuracy.<n>Neural Additive Experts (NAEs) employ a mixture of experts framework, learning multiple specialized networks per feature.<n>We show that NAEs achieve an optimal balance between predictive accuracy and transparent, feature-level explanations.
• Score: 45.48194499967696
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The trade-off between interpretability and accuracy remains a core challenge in machine learning. Standard Generalized Additive Models (GAMs) offer clear feature attributions but are often constrained by their strictly additive nature, which can limit predictive performance. Introducing feature interactions can boost accuracy yet may obscure individual feature contributions. To address these issues, we propose Neural Additive Experts (NAEs), a novel framework that seamlessly balances interpretability and accuracy. NAEs employ a mixture of experts framework, learning multiple specialized networks per feature, while a dynamic gating mechanism integrates information across features, thereby relaxing rigid additive constraints. Furthermore, we propose targeted regularization techniques to mitigate variance among expert predictions, facilitating a smooth transition from an exclusively additive model to one that captures intricate feature interactions while maintaining clarity in feature attributions. Our theoretical analysis and experiments on synthetic data illustrate the model's flexibility, and extensive evaluations on real-world datasets confirm that NAEs achieve an optimal balance between predictive accuracy and transparent, feature-level explanations. The code is available at https://github.com/Teddy-XiongGZ/NAE.

Related papers

• Sparsity and Superposition in Mixture of Experts [0.0]
  We show that MoE models cannot be explained mechanistically through the same lens.<n>We find that neither feature sparsity nor feature importance cause discontinuous phase changes.<n>We propose a new definition of expert specialization based on monosemantic feature representation rather than load balancing.
  arXiv  Detail & Related papers  (2025-10-26T22:44:35Z)
• Detecting and Pruning Prominent but Detrimental Neurons in Large Language Models [68.57424628540907]
  Large language models (LLMs) often develop learned mechanisms specialized to specific datasets.<n>We introduce a fine-tuning approach designed to enhance generalization by identifying and pruning neurons associated with dataset-specific mechanisms.<n>Our method employs Integrated Gradients to quantify each neuron's influence on high-confidence predictions, pinpointing those that disproportionately contribute to dataset-specific performance.
  arXiv  Detail & Related papers  (2025-07-12T08:10:10Z)
• Improving Network Interpretability via Explanation Consistency Evaluation [56.14036428778861]
  We propose a framework that acquires more explainable activation heatmaps and simultaneously increase the model performance. Specifically, our framework introduces a new metric, i.e., explanation consistency, to reweight the training samples adaptively in model learning. Our framework then promotes the model learning by paying closer attention to those training samples with a high difference in explanations.
  arXiv  Detail & Related papers  (2024-08-08T17:20:08Z)
• Intrinsic User-Centric Interpretability through Global Mixture of Experts [31.738009841932374]
  InterpretCC is a family of intrinsically interpretable neural networks that optimize for ease of human understanding and explanation faithfulness.<n>We show that InterpretCC explanations are found to have higher actionability and usefulness over other intrinsically interpretable approaches.
  arXiv  Detail & Related papers  (2024-02-05T11:55:50Z)
• LaPLACE: Probabilistic Local Model-Agnostic Causal Explanations [1.0370398945228227]
  We introduce LaPLACE-explainer, designed to provide probabilistic cause-and-effect explanations for machine learning models. The LaPLACE-Explainer component leverages the concept of a Markov blanket to establish statistical boundaries between relevant and non-relevant features. Our approach offers causal explanations and outperforms LIME and SHAP in terms of local accuracy and consistency of explained features.
  arXiv  Detail & Related papers  (2023-10-01T04:09:59Z)
• On the Joint Interaction of Models, Data, and Features [82.60073661644435]
  We introduce a new tool, the interaction tensor, for empirically analyzing the interaction between data and model through features. Based on these observations, we propose a conceptual framework for feature learning. Under this framework, the expected accuracy for a single hypothesis and agreement for a pair of hypotheses can both be derived in closed-form.
  arXiv  Detail & Related papers  (2023-06-07T21:35:26Z)
• Curve Your Enthusiasm: Concurvity Regularization in Differentiable Generalized Additive Models [5.519653885553456]
  Generalized Additive Models (GAMs) have recently experienced a resurgence in popularity due to their interpretability. We show how concurvity can severly impair the interpretability of GAMs. We propose a remedy: a conceptually simple, yet effective regularizer which penalizes pairwise correlations of the non-linearly transformed feature variables.
  arXiv  Detail & Related papers  (2023-05-19T06:55:49Z)
• ER: Equivariance Regularizer for Knowledge Graph Completion [107.51609402963072]
  We propose a new regularizer, namely, Equivariance Regularizer (ER) ER can enhance the generalization ability of the model by employing the semantic equivariance between the head and tail entities. The experimental results indicate a clear and substantial improvement over the state-of-the-art relation prediction methods.
  arXiv  Detail & Related papers  (2022-06-24T08:18:05Z)
• Interpretable Mixture of Experts [71.55701784196253]
  Interpretable Mixture of Experts (IME) is an inherently-interpretable modeling framework. IME is demonstrated to be more accurate than single interpretable models and perform comparably with existing state-of-the-art Deep Neural Networks (DNNs) in accuracy. IME's explanations are compared to commonly-used post-hoc explanations methods through a user study.
  arXiv  Detail & Related papers  (2022-06-05T06:40:15Z)
• Interpretable Additive Recurrent Neural Networks For Multivariate Clinical Time Series [4.125698836261585]
  We present the Interpretable-RNN (I-RNN) that balances model complexity and accuracy by forcing the relationship between variables in the model to be additive. I-RNN specifically captures the unique characteristics of clinical time series, which are unevenly sampled in time, asynchronously acquired, and have missing data. We evaluate the I-RNN model on the Physionet 2012 Challenge dataset to predict in-hospital mortality, and on a real-world clinical decision support task: predicting hemodynamic interventions in the intensive care unit.
  arXiv  Detail & Related papers  (2021-09-15T22:30:19Z)
• Generative Counterfactuals for Neural Networks via Attribute-Informed Perturbation [51.29486247405601]
  We design a framework to generate counterfactuals for raw data instances with the proposed Attribute-Informed Perturbation (AIP) By utilizing generative models conditioned with different attributes, counterfactuals with desired labels can be obtained effectively and efficiently. Experimental results on real-world texts and images demonstrate the effectiveness, sample quality as well as efficiency of our designed framework.
  arXiv  Detail & Related papers  (2021-01-18T08:37:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.