Regionally Additive Models: Explainable-by-design models minimizing feature interactions

• URL: http://arxiv.org/abs/2309.12215v1
• Date: Thu, 21 Sep 2023 16:16:22 GMT
• Title: Regionally Additive Models: Explainable-by-design models minimizing feature interactions
• Authors: Vasilis Gkolemis, Anargiros Tzerefos, Theodore Dalamagas, Eirini Ntoutsi, Christos Diou
• Abstract summary: Generalized Additive Models (GAMs) are widely used explainable-by-design models in various applications. In ML problems where the output depends on multiple features simultaneously, GAMs fail to capture the interaction terms of the underlying function. We propose Regionally Additive Models (RAMs), a novel class of explainable-by-design models.
• Score: 8.118449359076438
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generalized Additive Models (GAMs) are widely used explainable-by-design models in various applications. GAMs assume that the output can be represented as a sum of univariate functions, referred to as components. However, this assumption fails in ML problems where the output depends on multiple features simultaneously. In these cases, GAMs fail to capture the interaction terms of the underlying function, leading to subpar accuracy. To (partially) address this issue, we propose Regionally Additive Models (RAMs), a novel class of explainable-by-design models. RAMs identify subregions within the feature space where interactions are minimized. Within these regions, it is more accurate to express the output as a sum of univariate functions (components). Consequently, RAMs fit one component per subregion of each feature instead of one component per feature. This approach yields a more expressive model compared to GAMs while retaining interpretability. The RAM framework consists of three steps. Firstly, we train a black-box model. Secondly, using Regional Effect Plots, we identify subregions where the black-box model exhibits near-local additivity. Lastly, we fit a GAM component for each identified subregion. We validate the effectiveness of RAMs through experiments on both synthetic and real-world datasets. The results confirm that RAMs offer improved expressiveness compared to GAMs while maintaining interpretability.

Related papers

• Interpretability-by-Design with Accurate Locally Additive Models and Conditional Feature Effects [6.312016976793988]
  We propose emphConditionally Additive Local Models (CALMs)<n>CALMs balance interpretability of GAMs with the accuracy of GA$2$Ms.<n>Experiments show CALMs consistently outperform GAMs and achieve accuracy comparable with GA$2$Ms.
  arXiv  Detail & Related papers  (2026-02-18T14:45:33Z)
• Sparse Semantic Dimension as a Generalization Certificate for LLMs [53.681678236115836]
  We introduce the Sparse Semantic Dimension (SSD), a complexity measure derived from the active feature vocabulary of a Sparse Autoencoder (SAE) trained on the model's layers.<n>We validate this framework on GPT-2 Small and Gemma-2B, demonstrating that our bound provides non-vacuous certificates at realistic sample sizes.
  arXiv  Detail & Related papers  (2026-02-11T21:45:18Z)
• OpenInsGaussian: Open-vocabulary Instance Gaussian Segmentation with Context-aware Cross-view Fusion [89.98812408058336]
  We introduce textbfOpenInsGaussian, an textbfOpen-vocabulary textbfInstance textbfGaussian segmentation framework with Context-aware Cross-view Fusion.<n>OpenInsGaussian achieves state-of-the-art results in open-vocabulary 3D Gaussian segmentation, outperforming existing baselines by a large margin.
  arXiv  Detail & Related papers  (2025-10-21T03:24:12Z)
• LIRA: Inferring Segmentation in Large Multi-modal Models with Local Interleaved Region Assistance [56.474856189865946]
  Large multi-modal models (LMMs) struggle with inaccurate segmentation and hallucinated comprehension.<n>We propose LIRA, a framework that capitalizes on the complementary relationship between visual comprehension and segmentation.<n>LIRA achieves state-of-the-art performance in both segmentation and comprehension tasks.
  arXiv  Detail & Related papers  (2025-07-08T07:46:26Z)
• Robust Uplift Modeling with Large-Scale Contexts for Real-time Marketing [6.511772664252086]
  Uplift modeling is proposed to solve the problem, which applies different treatments (e.g., discounts, bonus) to satisfy corresponding users. In real-world scenarios, there are rich contexts available in the online platform (e.g., short videos, news) and the uplift model needs to infer an incentive for each user. We propose a novel model-agnostic Robust Uplift Modeling with Large-Scale Contexts (UMLC) framework for Real-time Marketing.
  arXiv  Detail & Related papers  (2025-01-04T08:55:50Z)
• Succinct Interaction-Aware Explanations [33.25637826682827]
  SHAP is a popular approach to explain black-box models by revealing the importance of individual features. NSHAP, on the other hand, reports the additive importance for all subsets of features. We propose to combine the best of these two worlds, by partitioning the features into parts that significantly interact.
  arXiv  Detail & Related papers  (2024-02-08T11:04:11Z)
• Sample Complexity Characterization for Linear Contextual MDPs [67.79455646673762]
  Contextual decision processes (CMDPs) describe a class of reinforcement learning problems in which the transition kernels and reward functions can change over time with different MDPs indexed by a context variable. CMDPs serve as an important framework to model many real-world applications with time-varying environments. We study CMDPs under two linear function approximation models: Model I with context-varying representations and common linear weights for all contexts; and Model II with common representations for all contexts and context-varying linear weights.
  arXiv  Detail & Related papers  (2024-02-05T03:25:04Z)
• NPEFF: Non-Negative Per-Example Fisher Factorization [52.44573961263344]
  We introduce a novel interpretability method called NPEFF that is readily applicable to any end-to-end differentiable model. We demonstrate that NPEFF has interpretable tunings through experiments on language and vision models.
  arXiv  Detail & Related papers  (2023-10-07T02:02:45Z)
• iBARLE: imBalance-Aware Room Layout Estimation [54.819085005591894]
  Room layout estimation predicts layouts from a single panorama. There are significant imbalances in real-world datasets including the dimensions of layout complexity, camera locations, and variation in scene appearance. We propose imBalance-Aware Room Layout Estimation (iBARLE) framework to address these issues. iBARLE consists of (1) Appearance Variation Generation (AVG) module, (2) Complex Structure Mix-up (CSMix) module, which enhances generalizability w.r.t. room structure, and (3) a gradient-based layout objective function.
  arXiv  Detail & Related papers  (2023-08-29T06:20:36Z)
• MALUNet: A Multi-Attention and Light-weight UNet for Skin Lesion Segmentation [13.456935850832565]
  We propose a light-weight model to achieve competitive performances for skin lesion segmentation at the lowest cost of parameters and computational complexity. We combine four modules with our U-shape architecture and obtain a light-weight medical image segmentation model dubbed as MALUNet. Compared with UNet, our model improves the mIoU and DSC metrics by 2.39% and 1.49%, respectively, with a 44x and 166x reduction in the number of parameters and computational complexity.
  arXiv  Detail & Related papers  (2022-11-03T13:19:22Z)
• Marginalized particle Gibbs for multiple state-space models coupled through shared parameters [18.45278329799526]
  Particle Gibbs (PG) samplers are an efficient class of algorithms for inference in SSMs. We present two different PG samplers that marginalize static model parameters on-the-fly. We show that they can be combined to form an efficient sampler for a model with strong dependencies between states and parameters.
  arXiv  Detail & Related papers  (2022-10-13T21:49:40Z)
• Aggregated Multi-output Gaussian Processes with Knowledge Transfer Across Domains [39.25639417233822]
  This article offers a multi-output Gaussian process (MoGP) model that infers functions for attributes using multiple aggregate datasets of respective granularities. Experiments demonstrate that the proposed model outperforms in the task of refining coarse-grained aggregate data on real-world datasets.
  arXiv  Detail & Related papers  (2022-06-24T08:07:20Z)
• A Probabilistic Hard Attention Model For Sequentially Observed Scenes [5.203329540700176]
  A visual hard attention model actively selects and observes a sequence of subregions in an image to make a prediction. In this paper, we design an efficient hard attention model for classifying such sequentially observed scenes. Our model gains 2-10% higher accuracy than the baseline models when both have seen only a couple of glimpses.
  arXiv  Detail & Related papers  (2021-11-15T04:47:47Z)
• Partial Order in Chaos: Consensus on Feature Attributions in the Rashomon Set [50.67431815647126]
  Post-hoc global/local feature attribution methods are being progressively employed to understand machine learning models. We show that partial orders of local/global feature importance arise from this methodology. We show that every relation among features present in these partial orders also holds in the rankings provided by existing approaches.
  arXiv  Detail & Related papers  (2021-10-26T02:53:14Z)
• From Sets to Multisets: Provable Variational Inference for Probabilistic Integer Submodular Models [82.95892656532696]
  Submodular functions have been studied extensively in machine learning and data mining. In this work, we propose a continuous DR-submodular extension for integer submodular functions. We formulate a new probabilistic model which is defined through integer submodular functions.
  arXiv  Detail & Related papers  (2020-06-01T22:20:45Z)
• Particle-Gibbs Sampling For Bayesian Feature Allocation Models [77.57285768500225]
  Most widely used MCMC strategies rely on an element wise Gibbs update of the feature allocation matrix. We have developed a Gibbs sampler that can update an entire row of the feature allocation matrix in a single move. This sampler is impractical for models with a large number of features as the computational complexity scales exponentially in the number of features. We develop a Particle Gibbs sampler that targets the same distribution as the row wise Gibbs updates, but has computational complexity that only grows linearly in the number of features.
  arXiv  Detail & Related papers  (2020-01-25T22:11:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.