LassoFlexNet: Flexible Neural Architecture for Tabular Data

• URL: http://arxiv.org/abs/2603.20631v1
• Date: Sat, 21 Mar 2026 03:59:47 GMT
• Title: LassoFlexNet: Flexible Neural Architecture for Tabular Data
• Abstract summary: We propose emphLassoFlexNet, an architecture that evaluates the linear and nonlinear marginal contribution of each input via Per-Feature Embeddings.<n>Across $52$ from three benchmarks, LassoFlexNet matches or outperforms leading tree-based models, achieving up to a $10$% relative gain.
• Score: 15.932140218953817
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Despite their dominance in vision and language, deep neural networks often underperform relative to tree-based models on tabular data. To bridge this gap, we incorporate five key inductive biases into deep learning: robustness to irrelevant features, axis alignment, localized irregularities, feature heterogeneity, and training stability. We propose \emph{LassoFlexNet}, an architecture that evaluates the linear and nonlinear marginal contribution of each input via Per-Feature Embeddings, and sparsely selects relevant variables using a Tied Group Lasso mechanism. Because these components introduce optimization challenges that destabilize standard proximal methods, we develop a \emph{Sequential Hierarchical Proximal Adaptive Gradient optimizer with exponential moving averages (EMA)} to ensure stable convergence. Across $52$ datasets from three benchmarks, LassoFlexNet matches or outperforms leading tree-based models, achieving up to a $10$\% relative gain, while maintaining Lasso-like interpretability. We substantiate these empirical results with ablation studies and theoretical proofs confirming the architecture's enhanced expressivity and structural breaking of undesired rotational invariance.

Related papers

• TabNSM: Neural Sparse Mixer for Tabular Regression [2.124023760378586]
  We propose TabNSM, a scalable regression framework that extends our earlier sparse-attention and mixer architectures.<n>At its core, the Adaptive Sparse Interaction Module (ASIM) integrates foreground feature discovery, sparse local interaction encoding, and Feature-Token Mixing.<n>For regression, TabNSM introduces three complementary components: a Multi-Stage Regression Head for progressive prediction refinement; GridLoss, an ordinal-aware soft-binning objective that incorporates target structure into representation learning; and RISE, a difficulty-aware sampling strategy based on loss-quantile bins.
  arXiv  Detail & Related papers  (2026-08-18T17:22:49Z)
• Joint Model and Data Sparsification via the Marginal Likelihood [53.29070892356214]
  We propose jointly learning individual feature and sample relevancies, enabling simultaneous model and data sparsification.<n>This symmetric pruning of model and data offers a natural extension that preserves conjugacy.<n> Empirical results across diverse regression tasks affirm that a joint ARD approach consistently yields both sparse and robust prediction models.
  arXiv  Detail & Related papers  (2026-05-28T13:26:53Z)
• Customer Churn Prediction on Structured Data Using FT-Transformer and Stacking Ensembles [0.0]
  This study introduces a validated hybrid architecture that integrates feature-tokenized transformers (FT-Transformer) with gradient-boosted trees through calibration stacking.<n>The proposed framework addresses persistent gaps in statistical validation, probability, calibration, and found in prior research.<n>On a public bank churn dataset, the hybrid model 62.10% F1, 0.861 AUC-ROC, and 0.647 PR-AUC, outperforming the Multi-Layer Perceptron (MLP) baseline by 3.37 F1 points and 0.027 AUC with 95% confidence intervals reported.
  arXiv  Detail & Related papers  (2026-05-26T23:02:38Z)
• Self-Improving Tabular Language Models via Iterative Group Alignment [76.43372236405664]
  We introduce TabGRAA, the first self-improving framework for data generation via automated feedback.<n>It partitions newly generated samples into high- and low-quality groups, then prioritizes a group-relative advantage objective.<n>Experiments show TabGRAA outperforms existing methods in fidelity, utility, and privacy.
  arXiv  Detail & Related papers  (2026-04-21T01:29:52Z)
• Closing the gap on tabular data with Fourier and Implicit Categorical Features [3.071430103942477]
  We show that our proposed feature preprocessing significantly boosts the performance of deep learning models.<n>We show that our proposed feature preprocessing enables them to achieve a performance that closely matches or surpasses XGBoost.
  arXiv  Detail & Related papers  (2026-02-26T16:40:23Z)
• Topologically-Stabilized Graph Neural Networks: Empirical Robustness Across Domains [0.0]
  Graph Neural Networks (GNNs) have become the standard for graph representation learning but remain vulnerable to structural perturbations.<n>We propose a novel framework that integrates persistent homology features with stability regularization to enhance robustness.<n>Our approach demonstrates exceptional robustness to edge perturbations while maintaining competitive accuracy.
  arXiv  Detail & Related papers  (2025-12-15T19:39:11Z)
• Beyond Softmax: A Natural Parameterization for Categorical Random Variables [61.709831225296305]
  We introduce the $textitcatnat$ function, a function composed of a sequence of hierarchical binary splits.<n>A rich set of experiments show that the proposed function improves the learning efficiency and yields models characterized by consistently higher test performance.
  arXiv  Detail & Related papers  (2025-09-29T12:55:50Z)
• Deep Hierarchical Learning with Nested Subspace Networks [53.71337604556311]
  We propose Nested Subspace Networks (NSNs) for large neural networks.<n>NSNs enable a single model to be dynamically and granularly adjusted across a continuous spectrum of compute budgets.<n>We show that NSNs can be surgically applied to pre-trained LLMs and unlock a smooth and predictable compute-performance frontier.
  arXiv  Detail & Related papers  (2025-09-22T15:13:14Z)
• Tuning for Trustworthiness -- Balancing Performance and Explanation Consistency in Neural Network Optimization [49.567092222782435]
  We introduce the novel concept of XAI consistency, defined as the agreement among different feature attribution methods.<n>We create a multi-objective optimization framework that balances predictive performance with explanation.<n>Our research provides a foundation for future investigations into whether models from the trade-off zone-balancing performance loss and XAI consistency-exhibit greater robustness.
  arXiv  Detail & Related papers  (2025-05-12T13:19:14Z)
• Consistency of augmentation graph and network approximability in contrastive learning [3.053989095162017]
  We analyze pointwise and spectral consistency of the augmentation graph Laplacian.<n>We show that Laplacian converges to a weighted Laplace-Beltrami operator on the natural data manifold.<n>These consistency results ensure that the graph Laplacian spectrum effectively captures the manifold geometry.
  arXiv  Detail & Related papers  (2025-02-06T18:55:51Z)
• The Risk of Federated Learning to Skew Fine-Tuning Features and Underperform Out-of-Distribution Robustness [50.52507648690234]
  Federated learning has the risk of skewing fine-tuning features and compromising the robustness of the model. We introduce three robustness indicators and conduct experiments across diverse robust datasets. Our approach markedly enhances the robustness across diverse scenarios, encompassing various parameter-efficient fine-tuning methods.
  arXiv  Detail & Related papers  (2024-01-25T09:18:51Z)
• Efficient Semi-Implicit Variational Inference [65.07058307271329]
  We propose an efficient and scalable semi-implicit extrapolational (SIVI) Our method maps SIVI's evidence to a rigorous inference of lower gradient values.
  arXiv  Detail & Related papers  (2021-01-15T11:39:09Z)
• Neural Ensemble Search for Uncertainty Estimation and Dataset Shift [67.57720300323928]
  Ensembles of neural networks achieve superior performance compared to stand-alone networks in terms of accuracy, uncertainty calibration and robustness to dataset shift. We propose two methods for automatically constructing ensembles with emphvarying architectures. We show that the resulting ensembles outperform deep ensembles not only in terms of accuracy but also uncertainty calibration and robustness to dataset shift.
  arXiv  Detail & Related papers  (2020-06-15T17:38:15Z)
• Residual Correlation in Graph Neural Network Regression [39.54530450932135]
  We show that conditional independence assumption severely limits predictive power. We address this problem with an interpretable and efficient framework. Our framework achieves substantially higher accuracy than competing baselines.
  arXiv  Detail & Related papers  (2020-02-19T16:32:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.