Iterative Feature Exclusion Ranking for Deep Tabular Learning

• URL: http://arxiv.org/abs/2412.16442v1
• Date: Sat, 21 Dec 2024 02:12:14 GMT
• Title: Iterative Feature Exclusion Ranking for Deep Tabular Learning
• Authors: Fathi Said Emhemed Shaninah, AbdulRahman M. A. Baraka, Mohd Halim Mohd Noor,
• Abstract summary: Tabular data is a common format for storing information in rows and columns to represent data entries and their features. Deep neural networks have become the main approach for modeling a wide range of domains including computer vision and NLP. Recent models featuring an internal feature selection mechanism with end-to-end gradient-based optimization fail to account for the contextual dependence of feature importance. The proposed module iteratively excludes each feature from the input data and computes the attention scores, which represent the impact of the features on the prediction.
• Score: 0.0
• License:
• Abstract: Tabular data is a common format for storing information in rows and columns to represent data entries and their features. Although deep neural networks have become the main approach for modeling a wide range of domains including computer vision and NLP, many of them are not well-suited for tabular data. Recently, a few deep learning models have been proposed for deep tabular learning, featuring an internal feature selection mechanism with end-to-end gradient-based optimization. However, their feature selection mechanisms are unidimensional, and hence fail to account for the contextual dependence of feature importance, potentially overlooking crucial interactions that govern complex tasks. In addition, they overlook the bias of high-impact features and the risk associated with the limitations of attention generalization. To address this limitation, this study proposes a novel iterative feature exclusion module that enhances the feature importance ranking in tabular data. The proposed module iteratively excludes each feature from the input data and computes the attention scores, which represent the impact of the features on the prediction. By aggregating the attention scores from each iteration, the proposed module generates a refined representation of feature importance that captures both global and local interactions between features. The effectiveness of the proposed module is evaluated on four public datasets. The results demonstrate that the proposed module consistently outperforms state-of-the-art methods and baseline models in feature ranking and classification tasks. The code is publicly available at https://github.com/abaraka2020/Iterative-Feature-Exclusion-Ranking-Module and https://github.com/mohalim/IFENet

Related papers

• TabSeq: A Framework for Deep Learning on Tabular Data via Sequential Ordering [5.946579489162407]
  This work introduces TabSeq, a novel framework for the sequential ordering of features. Finding the optimum sequence order for such features could improve the deep learning models' learning process.
  arXiv  Detail & Related papers  (2024-10-17T04:10:36Z)
• A Performance-Driven Benchmark for Feature Selection in Tabular Deep Learning [131.2910403490434]
  Data scientists typically collect as many features as possible into their datasets, and even engineer new features from existing ones. Existing benchmarks for tabular feature selection consider classical downstream models, toy synthetic datasets, or do not evaluate feature selectors on the basis of downstream performance. We construct a challenging feature selection benchmark evaluated on downstream neural networks including transformers. We also propose an input-gradient-based analogue of Lasso for neural networks that outperforms classical feature selection methods on challenging problems.
  arXiv  Detail & Related papers  (2023-11-10T05:26:10Z)
• Causal Feature Selection via Transfer Entropy [59.999594949050596]
  Causal discovery aims to identify causal relationships between features with observational data. We introduce a new causal feature selection approach that relies on the forward and backward feature selection procedures. We provide theoretical guarantees on the regression and classification errors for both the exact and the finite-sample cases.
  arXiv  Detail & Related papers  (2023-10-17T08:04:45Z)
• Deep networks for system identification: a Survey [56.34005280792013]
  System identification learns mathematical descriptions of dynamic systems from input-output data. Main aim of the identified model is to predict new data from previous observations. We discuss architectures commonly adopted in the literature, like feedforward, convolutional, and recurrent networks.
  arXiv  Detail & Related papers  (2023-01-30T12:38:31Z)
• Interpretability with full complexity by constraining feature information [1.52292571922932]
  Interpretability is a pressing issue for machine learning. We approach interpretability from a new angle: constrain the information about the features without restricting the complexity of the model. We develop a framework for extracting insight from the spectrum of approximate models.
  arXiv  Detail & Related papers  (2022-11-30T18:59:01Z)
• Beyond Importance Scores: Interpreting Tabular ML by Visualizing Feature Semantics [17.410093908967976]
  Interpretability is becoming an active research topic as machine learning (ML) models are more widely used to make critical decisions. Much of the existing interpretability methods used for tabular data only report feature-importance scores. We address this limitation by introducing Feature Vectors, a new global interpretability method.
  arXiv  Detail & Related papers  (2021-11-10T19:42:33Z)
• Learning Debiased and Disentangled Representations for Semantic Segmentation [52.35766945827972]
  We propose a model-agnostic and training scheme for semantic segmentation. By randomly eliminating certain class information in each training iteration, we effectively reduce feature dependencies among classes. Models trained with our approach demonstrate strong results on multiple semantic segmentation benchmarks.
  arXiv  Detail & Related papers  (2021-10-31T16:15:09Z)
• Towards Open-World Feature Extrapolation: An Inductive Graph Learning Approach [80.8446673089281]
  We propose a new learning paradigm with graph representation and learning. Our framework contains two modules: 1) a backbone network (e.g., feedforward neural nets) as a lower model takes features as input and outputs predicted labels; 2) a graph neural network as an upper model learns to extrapolate embeddings for new features via message passing over a feature-data graph built from observed data.
  arXiv  Detail & Related papers  (2021-10-09T09:02:45Z)
• Multi-scale Interactive Network for Salient Object Detection [91.43066633305662]
  We propose the aggregate interaction modules to integrate the features from adjacent levels. To obtain more efficient multi-scale features, the self-interaction modules are embedded in each decoder unit. Experimental results on five benchmark datasets demonstrate that the proposed method without any post-processing performs favorably against 23 state-of-the-art approaches.
  arXiv  Detail & Related papers  (2020-07-17T15:41:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.