Modeling Nonlinear Feature Interactions with Product-Unit Residual Networks

• URL: http://arxiv.org/abs/2606.06861v1
• Date: Fri, 05 Jun 2026 03:11:08 GMT
• Title: Modeling Nonlinear Feature Interactions with Product-Unit Residual Networks
• Abstract summary: Product-unit residual networks (PURe) integrate multiplicative product units with residual connections to explicitly model cross-feature couplings.<n>We conduct a systematic evaluation on an interaction-driven synthetic benchmark and two real-world datasets.<n> PURe achieves competitive or improved performance, better robustness and sample efficiency in low-data regimes, and enhanced interaction-level interpretability.
• Score: 0.9367224590861915
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Understanding nonlinear feature interactions is crucial in science and engineering, yet standard multilayer perceptrons (MLPs) often capture such interactions only implicitly, leading to entangled representations that can impair robustness and interpretability. We investigate product-unit residual networks (PURe) that integrate multiplicative product units with residual connections to explicitly model cross-feature couplings while stabilizing optimization. We conduct a systematic evaluation on an interaction-driven synthetic benchmark and two real-world datasets, assessing predictive accuracy, robustness to Gaussian feature noise, and performance under limited training data, and we compare real- and complex-valued variants under a matched parameter budget. Beyond accuracy, SHapley Additive exPlanations (SHAP)-based interaction analyses show that PURe learns more concentrated and structurally coherent interaction patterns than MLP baselines. Overall, PURe achieves competitive or improved performance, better robustness and sample efficiency in low-data regimes, and enhanced interaction-level interpretability.

Related papers

• A unified framework for global and local interpretability using adaptive derivative-ordered random explanation [2.6717072353469997]
  Post-hoc interpretability of complex machine learning models is of paramount importance, especially in real-world high-stakes domains such as healthcare and finance.<n>This paper provides a novel method - Adaptive Derivative-Ordered Random Explanation (ADORE) - that leverages first- and second-order derivatives to accommodate nonlinear model complexities.<n>ADORE integrates global feature importance with local sample contributions, precisely quantifying feature impact by capturing both magnitude and direction.<n>It achieves computational efficiency through randomized singular value decomposition (SVD) and dynamic sparsity detection, making it scalable to large, high-dimensional datasets.
  arXiv  Detail & Related papers  (2026-09-15T13:34:10Z)
• Beyond Isolation: Unlocking Reinforcement Learning Component Synergy for Sample-Efficient Continuous Control [59.15149762610843]
  Reinforcement learning systems are significantly more complex than other machine learning paradigms.<n>Despite advances in individual algorithmic components, their functional interdependencies remain underexplored.<n>We propose ROSER, an RL framework that coordinates three critical dimensions: Model-based Representation, Optimization Stability, and Experience Replay.
  arXiv  Detail & Related papers  (2026-08-07T10:38:31Z)
• Graph Structured Combinatorial Semi-Bandit with Nonlinear Reward Associations through Separable Signals [9.51084845180104]
  We develop novel generic and adaptive strategies equipped with routines for graph-based causal reward modeling.<n>We establish theoretical performance guarantees sublinear in time and linear in data volume over time.<n>We conduct numerical experiments using both benchmarked synthetic and real-world transportation datasets.
  arXiv  Detail & Related papers  (2026-06-12T17:11:56Z)
• SMI: Efficient Self-Supervised Learning via Mutual-Information-Inspired Dependency Optimization [8.864968998286772]
  Self-supervised learning (SSL) has achieved remarkable representation learning performance.<n>Many existing methods rely on large batch sizes, memory banks, momentum encoders, or global synchronization mechanisms.<n>We propose Semantic Mutual Information (SMI), a lightweight self-supervised objective derived from a mutual-information-inspired dependency formulation.
  arXiv  Detail & Related papers  (2026-06-06T20:50:04Z)
• Joint Model and Data Sparsification via the Marginal Likelihood [53.29070892356214]
  We propose jointly learning individual feature and sample relevancies, enabling simultaneous model and data sparsification.<n>This symmetric pruning of model and data offers a natural extension that preserves conjugacy.<n> Empirical results across diverse regression tasks affirm that a joint ARD approach consistently yields both sparse and robust prediction models.
  arXiv  Detail & Related papers  (2026-05-28T13:26:53Z)
• K-Models: a Flexible and Interpretable Method for Ordinal Clustering with Application to Antigen-Antibody Interaction Profiles [65.21921642886367]
  K-Models is a novel framework that integrates ordinal constraints and estimates key underlying elements of the random process generating the observed functional profiles.<n>The proposed method is evaluated through simulations and real-world applications.
  arXiv  Detail & Related papers  (2026-05-14T13:35:44Z)
• Max-pooling Network Revisited: Analyzing the Role of Semantic Probability in Multiple Instance Learning for Hallucination Detection [31.935824861650005]
  Hallucination detection has become increasingly important for improving the reliability of large language models.<n>Recently, hybrid approaches such as HaMI, which combine semantic consistency with internal model states via Multiple Instance Learning (MIL), have achieved state-of-the-art performance.<n>In this work, we first provide a theoretical analysis of HaMI in terms of decision margins, revealing that scaling internal states with semantic consistency leads to an enlarged decision margin.
  arXiv  Detail & Related papers  (2026-05-09T10:30:30Z)
• Explainability of Complex AI Models with Correlation Impact Ratio [10.61008729196936]
  Complex AI systems make better predictions but often lack transparency, limiting trustworthiness, interpretability, and safe deployment.<n>We introduce ExCIR (Explainability through Correlation Impact Ratio), a theoretically grounded, simple, and reliable metric for explaining the contribution of input features to model outputs.<n>We demonstrate that ExCIR captures dependencies arising from correlated features through a lightweight single pass formulation.
  arXiv  Detail & Related papers  (2026-01-10T21:56:24Z)
• SCALAR: Self-Calibrating Adaptive Latent Attention Representation Learning [2.152549977410309]
  High-dimensional, heterogeneous data with complex feature interactions pose significant challenges for traditional predictive modeling approaches.<n>Our architecture introduces an adaptive kernel-based attention mechanism that processes distinct feature groups separately before integration.<n> Experimental results show substantial improvements in performance metrics, compared to the state-of-the-art methods across diverse datasets.
  arXiv  Detail & Related papers  (2025-10-18T12:41:18Z)
• MaP: A Unified Framework for Reliable Evaluation of Pre-training Dynamics [72.00014675808228]
  Instability in Large Language Models evaluation process obscures true learning dynamics.<n>We introduce textbfMaP, a framework that integrates underlineMerging underlineand the underlinePass@k metric.<n>Experiments show that MaP yields significantly smoother performance curves, reduces inter-run variance, and ensures more consistent rankings.
  arXiv  Detail & Related papers  (2025-10-10T11:40:27Z)
• Tuning for Trustworthiness -- Balancing Performance and Explanation Consistency in Neural Network Optimization [49.567092222782435]
  We introduce the novel concept of XAI consistency, defined as the agreement among different feature attribution methods.<n>We create a multi-objective optimization framework that balances predictive performance with explanation.<n>Our research provides a foundation for future investigations into whether models from the trade-off zone-balancing performance loss and XAI consistency-exhibit greater robustness.
  arXiv  Detail & Related papers  (2025-05-12T13:19:14Z)
• iLOCO: Distribution-Free Inference for Feature Interactions [4.56754610152086]
  We develop a new model-agnostic metric for measuring the importance of pairwise feature interactions.<n>We also introduce an ensemble learning method for calculating the iLOCO metric and confidence intervals.<n>We validate our iLOCO metric and our confidence intervals on both synthetic and real data sets.
  arXiv  Detail & Related papers  (2025-02-10T16:49:46Z)
• Efficient and Interpretable Neural Networks Using Complex Lehmer Transform [11.095723123836965]
  We propose an efficient and interpretable neural network with a novel activation function called the weighted Lehmer transform.<n>We analyze the mathematical properties of both real-valued and complex-valued Lehmer activation units.<n> Empirical evaluations demonstrate that our proposed neural network achieves competitive accuracy on benchmark datasets.
  arXiv  Detail & Related papers  (2025-01-25T14:08:30Z)
• Exploiting Structure in Offline Multi-Agent RL: The Benefits of Low Interaction Rank [52.831993899183416]
  We introduce a structural assumption -- the interaction rank -- and establish that functions with low interaction rank are significantly more robust to distribution shift compared to general ones. We demonstrate that utilizing function classes with low interaction rank, when combined with regularization and no-regret learning, admits decentralized, computationally and statistically efficient learning in offline MARL.
  arXiv  Detail & Related papers  (2024-10-01T22:16:22Z)
• Out-of-distribution Generalization via Partial Feature Decorrelation [72.96261704851683]
  We present a novel Partial Feature Decorrelation Learning (PFDL) algorithm, which jointly optimize a feature decomposition network and the target image classification model. The experiments on real-world datasets demonstrate that our method can improve the backbone model's accuracy on OOD image classification datasets.
  arXiv  Detail & Related papers  (2020-07-30T05:48:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.