Understanding DNNs in Feature Interaction Models: A Dimensional Collapse Perspective

• URL: http://arxiv.org/abs/2604.26489v1
• Date: Wed, 29 Apr 2026 09:52:19 GMT
• Title: Understanding DNNs in Feature Interaction Models: A Dimensional Collapse Perspective
• Abstract summary: We present a novel perspective to understand the effectiveness of DNNs.<n>Our evaluation encompasses an overall study of complete DNN on two feature interaction models.<n> Experimental results demonstrate that both parallel and stacked DNNs can effectively mitigate the dimensional collapse of embeddings.
• Score: 45.50697497028273
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: DNNs have gained widespread adoption in feature interaction recommendation models. However, there has been a longstanding debate on their roles. On one hand, some works claim that DNNs possess the ability to implicitly capture high-order feature interactions. Conversely, recent studies have highlighted the limitations of DNNs in effectively learning dot products, specifically second-order interactions, let alone higher-order interactions. In this paper, we present a novel perspective to understand the effectiveness of DNNs: their impact on the dimensional robustness of the representations. In particular, we conduct extensive experiments involving both parallel DNNs and stacked DNNs. Our evaluation encompasses an overall study of complete DNN on two feature interaction models, alongside a fine-grained ablation analysis of components within DNNs. Experimental results demonstrate that both parallel and stacked DNNs can effectively mitigate the dimensional collapse of embeddings. Furthermore, a gradient-based theoretical analysis, supported by empirical evidence, uncovers the underlying mechanisms of dimensional collapse.

Related papers

• Joint Universal Adversarial Perturbations with Interpretations [19.140429650679593]
  In this paper, we propose a novel attacking framework to generate joint universal adversarial perturbations (JUAP) To the best of our knowledge, this is the first effort to study UAP for jointly attacking both DNNs and interpretations.
  arXiv  Detail & Related papers  (2024-08-03T08:58:04Z)
• Towards the Dynamics of a DNN Learning Symbolic Interactions [20.493304123269446]
  A series of theorems have been proven in recent years to show that for a given input sample, a small set of interactions between input variables can be considered as primitive inference patterns. This study proves the two-phase dynamics of interactions, providing a theoretical mechanism for how the generalization power of a DNN changes during the training process.
  arXiv  Detail & Related papers  (2024-07-27T07:34:49Z)
• Causal inference through multi-stage learning and doubly robust deep neural networks [10.021381302215062]
  Deep neural networks (DNNs) have demonstrated remarkable empirical performance in large-scale supervised learning problems. This study delves into the application of DNNs across a wide spectrum of intricate causal inference tasks.
  arXiv  Detail & Related papers  (2024-07-11T14:47:44Z)
• Two-Phase Dynamics of Interactions Explains the Starting Point of a DNN Learning Over-Fitted Features [68.3512123520931]
  We investigate the dynamics of a deep neural network (DNN) learning interactions. In this paper, we discover the DNN learns interactions in two phases. The first phase mainly penalizes interactions of medium and high orders, and the second phase mainly learns interactions of gradually increasing orders.
  arXiv  Detail & Related papers  (2024-05-16T17:13:25Z)
• Unveiling and Mitigating Generalized Biases of DNNs through the Intrinsic Dimensions of Perceptual Manifolds [46.47992213722412]
  Building fair deep neural networks (DNNs) is a crucial step towards achieving trustworthy artificial intelligence. We propose Intrinsic Dimension Regularization (IDR), which enhances the fairness and performance of models. In various image recognition benchmark tests, IDR significantly mitigates model bias while improving its performance.
  arXiv  Detail & Related papers  (2024-04-22T04:16:40Z)
• Transferability of coVariance Neural Networks and Application to Interpretable Brain Age Prediction using Anatomical Features [119.45320143101381]
  Graph convolutional networks (GCN) leverage topology-driven graph convolutional operations to combine information across the graph for inference tasks. We have studied GCNs with covariance matrices as graphs in the form of coVariance neural networks (VNNs) VNNs inherit the scale-free data processing architecture from GCNs and here, we show that VNNs exhibit transferability of performance over datasets whose covariance matrices converge to a limit object.
  arXiv  Detail & Related papers  (2023-05-02T22:15:54Z)
• On the Relationship Between Adversarial Robustness and Decision Region in Deep Neural Networks [24.82243003178264]
  We study the internal properties of Deep Neural Networks (DNNs) that affect model robustness under adversarial attacks.<n>We propose the novel concept of the Populated Region Set (PRS), where training samples are populated more frequently.
  arXiv  Detail & Related papers  (2022-07-07T16:06:34Z)
• On the Intrinsic Structures of Spiking Neural Networks [66.57589494713515]
  Recent years have emerged a surge of interest in SNNs owing to their remarkable potential to handle time-dependent and event-driven data. There has been a dearth of comprehensive studies examining the impact of intrinsic structures within spiking computations. This work delves deep into the intrinsic structures of SNNs, by elucidating their influence on the expressivity of SNNs.
  arXiv  Detail & Related papers  (2022-06-21T09:42:30Z)
• Generalization ability and Vulnerabilities to adversarial perturbations: Two sides of the same coin [1.7314342339585087]
  We use the self-organizing map (SOM) to analyze internal codes associated with deep learning models' decision-making. Our analyses suggest that shallow layers close to the input layer map onto homogeneous codes and that deep layers close to the output layer transform these homogeneous codes in shallow layers to diverse codes.
  arXiv  Detail & Related papers  (2022-05-22T23:14:27Z)
• Deep Neural Networks Are Congestion Games: From Loss Landscape to Wardrop Equilibrium and Beyond [12.622643370707328]
  We argue that our work provides a very promising novel tool for analyzing the deep neural networks (DNNs) We show how one can benefit from the classic readily available results from the latter when analyzing the former.
  arXiv  Detail & Related papers  (2020-10-21T14:11:40Z)
• Architecture Disentanglement for Deep Neural Networks [174.16176919145377]
  We introduce neural architecture disentanglement (NAD) to explain the inner workings of deep neural networks (DNNs) NAD learns to disentangle a pre-trained DNN into sub-architectures according to independent tasks, forming information flows that describe the inference processes. Results show that misclassified images have a high probability of being assigned to task sub-architectures similar to the correct ones.
  arXiv  Detail & Related papers  (2020-03-30T08:34:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.