Deep Feature Embedding for Tabular Data

• URL: http://arxiv.org/abs/2408.17162v1
• Date: Fri, 30 Aug 2024 10:05:24 GMT
• Title: Deep Feature Embedding for Tabular Data
• Authors: Yuqian Wu, Hengyi Luo, Raymond S. T. Lee,
• Abstract summary: This paper proposes a novel deep embedding framework with leverages lightweight deep neural networks. For numerical features, a two-step feature expansion and deep transformation technique is used to capture copious semantic information. Experiments are conducted on real-world datasets for performance evaluation.
• Score: 2.1301560294088318
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Tabular data learning has extensive applications in deep learning but its existing embedding techniques are limited in numerical and categorical features such as the inability to capture complex relationships and engineering. This paper proposes a novel deep embedding framework with leverages lightweight deep neural networks to generate effective feature embeddings for tabular data in machine learning research. For numerical features, a two-step feature expansion and deep transformation technique is used to capture copious semantic information. For categorical features, a unique identification vector for each entity is referred by a compact lookup table with a parameterized deep embedding function to uniform the embedding size dimensions, and transformed into a embedding vector using deep neural network. Experiments are conducted on real-world datasets for performance evaluation.

Related papers

• A Performance-Driven Benchmark for Feature Selection in Tabular Deep Learning [131.2910403490434]
  Data scientists typically collect as many features as possible into their datasets, and even engineer new features from existing ones. Existing benchmarks for tabular feature selection consider classical downstream models, toy synthetic datasets, or do not evaluate feature selectors on the basis of downstream performance. We construct a challenging feature selection benchmark evaluated on downstream neural networks including transformers. We also propose an input-gradient-based analogue of Lasso for neural networks that outperforms classical feature selection methods on challenging problems.
  arXiv  Detail & Related papers  (2023-11-10T05:26:10Z)
• On Characterizing the Evolution of Embedding Space of Neural Networks using Algebraic Topology [9.537910170141467]
  We study how the topology of feature embedding space changes as it passes through the layers of a well-trained deep neural network (DNN) through Betti numbers. We demonstrate that as depth increases, a topologically complicated dataset is transformed into a simple one, resulting in Betti numbers attaining their lowest possible value.
  arXiv  Detail & Related papers  (2023-11-08T10:45:12Z)
• Understanding Deep Representation Learning via Layerwise Feature Compression and Discrimination [33.273226655730326]
  We show that each layer of a deep linear network progressively compresses within-class features at a geometric rate and discriminates between-class features at a linear rate. This is the first quantitative characterization of feature evolution in hierarchical representations of deep linear networks.
  arXiv  Detail & Related papers  (2023-11-06T09:00:38Z)
• Multiclass classification for multidimensional functional data through deep neural networks [0.22843885788439797]
  We introduce a novel functional deep neural network (mfDNN) as an innovative data mining classification tool. We consider sparse deep neural network architecture with linear unit (ReLU) activation function and minimize the cross-entropy loss in the multiclass classification setup. We demonstrate the performance of mfDNN on simulated data and several benchmark datasets from different application domains.
  arXiv  Detail & Related papers  (2023-05-22T16:56:01Z)
• Transfer Learning with Deep Tabular Models [66.67017691983182]
  We show that upstream data gives tabular neural networks a decisive advantage over GBDT models. We propose a realistic medical diagnosis benchmark for tabular transfer learning. We propose a pseudo-feature method for cases where the upstream and downstream feature sets differ.
  arXiv  Detail & Related papers  (2022-06-30T14:24:32Z)
• DANets: Deep Abstract Networks for Tabular Data Classification and Regression [9.295859461145783]
  Abstract Layer (AbstLay) learns to explicitly group correlative input features and generate higher-level features for semantics abstraction. Family of Deep Abstract Networks (DANets) for tabular data classification and regression.
  arXiv  Detail & Related papers  (2021-12-06T12:15:28Z)
• Dive into Layers: Neural Network Capacity Bounding using Algebraic Geometry [55.57953219617467]
  We show that the learnability of a neural network is directly related to its size. We use Betti numbers to measure the topological geometric complexity of input data and the neural network. We perform the experiments on a real-world dataset MNIST and the results verify our analysis and conclusion.
  arXiv  Detail & Related papers  (2021-09-03T11:45:51Z)
• Rank-R FNN: A Tensor-Based Learning Model for High-Order Data Classification [69.26747803963907]
  Rank-R Feedforward Neural Network (FNN) is a tensor-based nonlinear learning model that imposes Canonical/Polyadic decomposition on its parameters. First, it handles inputs as multilinear arrays, bypassing the need for vectorization, and can thus fully exploit the structural information along every data dimension. We establish the universal approximation and learnability properties of Rank-R FNN, and we validate its performance on real-world hyperspectral datasets.
  arXiv  Detail & Related papers  (2021-04-11T16:37:32Z)
• Representation Extraction and Deep Neural Recommendation for Collaborative Filtering [9.367612782346207]
  This paper investigates the usage of novel representation learning algorithms to extract users and items representations from rating matrix. We propose a modular algorithm consisted of two main phases: REpresentation eXtraction and a deep neural NETwork (RexNet) RexNet is not dependent on unstructured auxiliary data such as visual and textual information, instead, it uses only the user-item rate matrix as its input.
  arXiv  Detail & Related papers  (2020-12-09T11:15:23Z)
• Dual-constrained Deep Semi-Supervised Coupled Factorization Network with Enriched Prior [80.5637175255349]
  We propose a new enriched prior based Dual-constrained Deep Semi-Supervised Coupled Factorization Network, called DS2CF-Net. To ex-tract hidden deep features, DS2CF-Net is modeled as a deep-structure and geometrical structure-constrained neural network. Our network can obtain state-of-the-art performance for representation learning and clustering.
  arXiv  Detail & Related papers  (2020-09-08T13:10:21Z)
• ReMarNet: Conjoint Relation and Margin Learning for Small-Sample Image Classification [49.87503122462432]
  We introduce a novel neural network termed Relation-and-Margin learning Network (ReMarNet) Our method assembles two networks of different backbones so as to learn the features that can perform excellently in both of the aforementioned two classification mechanisms. Experiments on four image datasets demonstrate that our approach is effective in learning discriminative features from a small set of labeled samples.
  arXiv  Detail & Related papers  (2020-06-27T13:50:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.