Infinite-Dimensional Feature Interaction

• URL: http://arxiv.org/abs/2405.13972v4
• Date: Sat, 02 Nov 2024 22:21:05 GMT
• Title: Infinite-Dimensional Feature Interaction
• Authors: Chenhui Xu, Fuxun Yu, Maoliang Li, Zihao Zheng, Zirui Xu, Jinjun Xiong, Xiang Chen,
• Abstract summary: InfiNet is a model architecture that enables feature interaction within an infinite-dimensional space created by RBF kernel. Our experiments reveal that InfiNet achieves new state-of-the-art, owing to its capability to leverage infinite-dimensional interactions.
• Score: 22.694922451810495
• License:
• Abstract: The past neural network design has largely focused on feature representation space dimension and its capacity scaling (e.g., width, depth), but overlooked the feature interaction space scaling. Recent advancements have shown shifted focus towards element-wise multiplication to facilitate higher-dimensional feature interaction space for better information transformation. Despite this progress, multiplications predominantly capture low-order interactions, thus remaining confined to a finite-dimensional interaction space. To transcend this limitation, classic kernel methods emerge as a promising solution to engage features in an infinite-dimensional space. We introduce InfiNet, a model architecture that enables feature interaction within an infinite-dimensional space created by RBF kernel. Our experiments reveal that InfiNet achieves new state-of-the-art, owing to its capability to leverage infinite-dimensional interactions, significantly enhancing model performance.

Related papers

• Contrastive Sequential Interaction Network Learning on Co-Evolving Riemannian Spaces [44.175106851212874]
  We propose a novel Contrastive model for Sequential Interaction Network learning on Co-Evolving RiEmannian spaces, CSINCERE. In CSINCERE, we formulate a Cross-Space aggregation for message-passing across representation spaces. We also design a Neural Curvature Estimator based on Ricci curvatures for modeling the space evolvement over time.
  arXiv  Detail & Related papers  (2024-01-02T15:19:01Z)
• Interpreting the Curse of Dimensionality from Distance Concentration and Manifold Effect [0.6906005491572401]
  We first summarize five challenges associated with manipulating high-dimensional data. We then delve into two major causes of the curse of dimensionality, distance concentration and manifold effect. By interpreting the causes of the curse of dimensionality, we can better understand the limitations of current models and algorithms.
  arXiv  Detail & Related papers  (2023-12-31T08:22:51Z)
• Dual Aggregation Transformer for Image Super-Resolution [92.41781921611646]
  We propose a novel Transformer model, Dual Aggregation Transformer, for image SR. Our DAT aggregates features across spatial and channel dimensions, in the inter-block and intra-block dual manner. Our experiments show that our DAT surpasses current methods.
  arXiv  Detail & Related papers  (2023-08-07T07:39:39Z)
• SINCERE: Sequential Interaction Networks representation learning on Co-Evolving RiEmannian manifolds [9.710773626459718]
  SINCERE is a novel method representing Sequential Interaction Networks on Co-Evolving RiEmannian manifold. It takes the user and item embedding trajectories in respective spaces into account, and emphasizes on the space evolvement that how curvature changes over time. Experiments on several real-world datasets demonstrate the promising performance of SINCERE over the state-of-the-art sequential interaction prediction methods.
  arXiv  Detail & Related papers  (2023-05-06T00:38:29Z)
• Neural Set Function Extensions: Learning with Discrete Functions in High Dimensions [63.21838830509772]
  We develop a framework for extending set functions onto low-dimensional continuous domains. Our framework subsumes many well-known extensions as special cases. We convert low-dimensional neural network bottlenecks into representations in high-dimensional spaces.
  arXiv  Detail & Related papers  (2022-08-08T10:58:02Z)
• Interaction Transformer for Human Reaction Generation [61.22481606720487]
  We propose a novel interaction Transformer (InterFormer) consisting of a Transformer network with both temporal and spatial attentions. Our method is general and can be used to generate more complex and long-term interactions.
  arXiv  Detail & Related papers  (2022-07-04T19:30:41Z)
• Eigenspace Restructuring: a Principle of Space and Frequency in Neural Networks [11.480563447698172]
  We show that the eigenstructure of infinite-width multilayer perceptrons (MLPs) depends solely on the concept frequency. We show that the topologies from deep convolutional networks (CNNs) restructure the associated eigenspaces into finer subspaces. The resulting fine-grained eigenstructure dramatically improves the network's learnability.
  arXiv  Detail & Related papers  (2021-12-10T15:44:14Z)
• Pure Exploration in Kernel and Neural Bandits [90.23165420559664]
  We study pure exploration in bandits, where the dimension of the feature representation can be much larger than the number of arms. To overcome the curse of dimensionality, we propose to adaptively embed the feature representation of each arm into a lower-dimensional space.
  arXiv  Detail & Related papers  (2021-06-22T19:51:59Z)
• Convolutions for Spatial Interaction Modeling [9.408751013132624]
  We consider the problem of spatial interaction modeling in the context of predicting the motion of actors around autonomous vehicles. We revisit convolutions and show that they can demonstrate comparable performance to graph networks in modeling spatial interactions with lower latency.
  arXiv  Detail & Related papers  (2021-04-15T00:41:30Z)
• A Spatial-Temporal Attentive Network with Spatial Continuity for Trajectory Prediction [74.00750936752418]
  We propose a novel model named spatial-temporal attentive network with spatial continuity (STAN-SC) First, spatial-temporal attention mechanism is presented to explore the most useful and important information. Second, we conduct a joint feature sequence based on the sequence and instant state information to make the generative trajectories keep spatial continuity.
  arXiv  Detail & Related papers  (2020-03-13T04:35:50Z)
• Neural Operator: Graph Kernel Network for Partial Differential Equations [57.90284928158383]
  This work is to generalize neural networks so that they can learn mappings between infinite-dimensional spaces (operators) We formulate approximation of the infinite-dimensional mapping by composing nonlinear activation functions and a class of integral operators. Experiments confirm that the proposed graph kernel network does have the desired properties and show competitive performance compared to the state of the art solvers.
  arXiv  Detail & Related papers  (2020-03-07T01:56:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.