Linear Iterative Feature Embedding: An Ensemble Framework for Interpretable Model

• URL: http://arxiv.org/abs/2103.09983v1
• Date: Thu, 18 Mar 2021 02:01:17 GMT
• Title: Linear Iterative Feature Embedding: An Ensemble Framework for Interpretable Model
• Authors: Agus Sudjianto, Jinwen Qiu, Miaoqi Li and Jie Chen
• Abstract summary: A new ensemble framework for interpretable model called Linear Iterative Feature Embedding (LIFE) has been developed. LIFE is able to fit a wide single-hidden-layer neural network (NN) accurately with three steps. LIFE consistently outperforms directly trained single-hidden-layer NNs and also outperforms many other benchmark models.
• Score: 6.383006473302968
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A new ensemble framework for interpretable model called Linear Iterative Feature Embedding (LIFE) has been developed to achieve high prediction accuracy, easy interpretation and efficient computation simultaneously. The LIFE algorithm is able to fit a wide single-hidden-layer neural network (NN) accurately with three steps: defining the subsets of a dataset by the linear projections of neural nodes, creating the features from multiple narrow single-hidden-layer NNs trained on the different subsets of the data, combining the features with a linear model. The theoretical rationale behind LIFE is also provided by the connection to the loss ambiguity decomposition of stack ensemble methods. Both simulation and empirical experiments confirm that LIFE consistently outperforms directly trained single-hidden-layer NNs and also outperforms many other benchmark models, including multi-layers Feed Forward Neural Network (FFNN), Xgboost, and Random Forest (RF) in many experiments. As a wide single-hidden-layer NN, LIFE is intrinsically interpretable. Meanwhile, both variable importance and global main and interaction effects can be easily created and visualized. In addition, the parallel nature of the base learner building makes LIFE computationally efficient by leveraging parallel computing.

Related papers

• ParsNets: A Parsimonious Orthogonal and Low-Rank Linear Networks for Zero-Shot Learning [22.823915322924304]
  This paper provides a novel parsimonious yet efficient design for zero-shot learning (ZSL), dubbed ParsNets, to achieve equivalent or even better performance against existing deep models. To facilitate the generalization of local linearities, we construct a maximal margin geometry on the learned features by enforcing low-rank constraints on intra-class samples and high-rank constraints on inter-class samples. To enhance the model's adaptability and counterbalance over/under-fittings in ZSL, a set of sample-wise indicators is employed to select a sparse subset from these base linear networks to form a composite
  arXiv  Detail & Related papers  (2023-12-15T11:32:11Z)
• Spline-based neural network interatomic potentials: blending classical and machine learning models [0.0]
  We introduce a new MLIP framework which blends the simplicity of spline-based MEAM potentials with the flexibility of a neural network architecture. We demonstrate how this framework can be used to probe the boundary between classical and ML IPs.
  arXiv  Detail & Related papers  (2023-10-04T15:42:26Z)
• Simplicity Bias in 1-Hidden Layer Neural Networks [28.755809186616702]
  Recent works have demonstrated that neural networks exhibit extreme simplicity bias(SB) We define SB as the network essentially being a function of a low dimensional projection of the inputs. We show that models trained on real datasets such as Imagenette and Waterbirds-Landbirds indeed depend on a low dimensional projection of the inputs.
  arXiv  Detail & Related papers  (2023-02-01T14:00:35Z)
• Learning an Invertible Output Mapping Can Mitigate Simplicity Bias in Neural Networks [66.76034024335833]
  We investigate why diverse/ complex features are learned by the backbone, and their brittleness is due to the linear classification head relying primarily on the simplest features. We propose Feature Reconstruction Regularizer (FRR) to ensure that the learned features can be reconstructed back from the logits. We demonstrate up to 15% gains in OOD accuracy on the recently introduced semi-synthetic datasets with extreme distribution shifts.
  arXiv  Detail & Related papers  (2022-10-04T04:01:15Z)
• Comparative Analysis of Interval Reachability for Robust Implicit and Feedforward Neural Networks [64.23331120621118]
  We use interval reachability analysis to obtain robustness guarantees for implicit neural networks (INNs) INNs are a class of implicit learning models that use implicit equations as layers. We show that our approach performs at least as well as, and generally better than, applying state-of-the-art interval bound propagation methods to INNs.
  arXiv  Detail & Related papers  (2022-04-01T03:31:27Z)
• SPINE: Soft Piecewise Interpretable Neural Equations [0.0]
  Fully connected networks are ubiquitous but uninterpretable. This paper takes a novel approach to piecewise fits by using set operations on individual pieces(parts) It can find a variety of applications where fully connected layers must be replaced by interpretable layers.
  arXiv  Detail & Related papers  (2021-11-20T16:18:00Z)
• Gone Fishing: Neural Active Learning with Fisher Embeddings [55.08537975896764]
  There is an increasing need for active learning algorithms that are compatible with deep neural networks. This article introduces BAIT, a practical representation of tractable, and high-performing active learning algorithm for neural networks.
  arXiv  Detail & Related papers  (2021-06-17T17:26:31Z)
• Fully differentiable model discovery [0.0]
  We propose an approach by combining neural network based surrogates with Sparse Bayesian Learning. Our work expands PINNs to various types of neural network architectures, and connects neural network-based surrogates to the rich field of Bayesian parameter inference.
  arXiv  Detail & Related papers  (2021-06-09T08:11:23Z)
• Rank-R FNN: A Tensor-Based Learning Model for High-Order Data Classification [69.26747803963907]
  Rank-R Feedforward Neural Network (FNN) is a tensor-based nonlinear learning model that imposes Canonical/Polyadic decomposition on its parameters. First, it handles inputs as multilinear arrays, bypassing the need for vectorization, and can thus fully exploit the structural information along every data dimension. We establish the universal approximation and learnability properties of Rank-R FNN, and we validate its performance on real-world hyperspectral datasets.
  arXiv  Detail & Related papers  (2021-04-11T16:37:32Z)
• PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning [109.84770951839289]
  We present PredRNN, a new recurrent network for learning visual dynamics from historical context. We show that our approach obtains highly competitive results on three standard datasets.
  arXiv  Detail & Related papers  (2021-03-17T08:28:30Z)
• Exploiting Heterogeneity in Operational Neural Networks by Synaptic Plasticity [87.32169414230822]
  Recently proposed network model, Operational Neural Networks (ONNs), can generalize the conventional Convolutional Neural Networks (CNNs) In this study the focus is drawn on searching the best-possible operator set(s) for the hidden neurons of the network based on the Synaptic Plasticity paradigm that poses the essential learning theory in biological neurons. Experimental results over highly challenging problems demonstrate that the elite ONNs even with few neurons and layers can achieve a superior learning performance than GIS-based ONNs.
  arXiv  Detail & Related papers  (2020-08-21T19:03:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.