CTR-KAN: KAN for Adaptive High-Order Feature Interaction Modeling

• URL: http://arxiv.org/abs/2408.08713v4
• Date: Sat, 25 Jan 2025 03:14:35 GMT
• Title: CTR-KAN: KAN for Adaptive High-Order Feature Interaction Modeling
• Authors: Yunxiao Shi, Wujiang Xu, Haimin Zhang, Qiang Wu, Yongfeng Zhang, Min Xu,
• Abstract summary: CTR-KAN is an adaptive framework for efficient high-order feature interaction modeling.<n>It builds upon the Kolmogorov-Arnold Network (KAN) paradigm, addressing its limitations in CTR prediction tasks.<n>CTR-KAN achieves state-of-the-art predictive accuracy with significantly lower computational costs.
• Score: 37.80127625183842
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Modeling high-order feature interactions is critical for click-through rate (CTR) prediction, yet traditional approaches often face challenges in balancing predictive accuracy and computational efficiency. These methods typically rely on pre-defined interaction orders, which limit flexibility and require extensive prior knowledge. Moreover, explicitly modeling high-order interactions can lead to significant computational overhead. To tackle these challenges, we propose CTR-KAN, an adaptive framework for efficient high-order feature interaction modeling. CTR-KAN builds upon the Kolmogorov-Arnold Network (KAN) paradigm, addressing its limitations in CTR prediction tasks. Specifically, we introduce key enhancements, including a lightweight architecture that reduces the computational complexity of KAN and supports embedding-based feature representations. Additionally, CTR-KAN integrates guided symbolic regression to effectively capture multiplicative relationships, a known challenge in standard KAN implementations. Extensive experiments demonstrate that CTR-KAN achieves state-of-the-art predictive accuracy with significantly lower computational costs. Its sparse network structure also facilitates feature pruning and enhances global interpretability, making CTR-KAN a powerful tool for efficient inference in real-world CTR prediction scenarios.

Related papers

• Mitigating Attention Hacking in Preference-Based Reward Modeling via Interaction Distillation [40.79564929465515]
  "Interaction Distillation" is a novel training framework for more adequate preference modeling through attention-level optimization.<n>It provides more stable and generalizable reward signals compared to state-of-the-art RM optimization methods.
  arXiv  Detail & Related papers  (2025-08-04T17:06:23Z)
• DLF: Enhancing Explicit-Implicit Interaction via Dynamic Low-Order-Aware Fusion for CTR Prediction [71.41414150295702]
  We propose a novel framework, Dynamic Low-Order-Aware Fusion (DLF), for modeling click-through rate (CTR) prediction.<n>RLI preserves low-order signals while mitigating redundancy from residual connections, and NAF dynamically integrates explicit and implicit representations at each layer, enhancing information sharing.<n>Experiments on public datasets demonstrate that DLF achieves state-of-the-art performance in CTR prediction, addressing key limitations of existing models.
  arXiv  Detail & Related papers  (2025-05-25T15:05:00Z)
• Scaled Supervision is an Implicit Lipschitz Regularizer [32.41225209639384]
  In social media, recommender systems rely on the click-through rate (CTR) as the standard metric to evaluate user engagement. We show that scaling supervision bandwidth can act as an implicit Lipschitz regularizer, stably optimizing existing CTR models to achieve better generalizability.
  arXiv  Detail & Related papers  (2025-03-19T01:01:28Z)
• Context-Preserving Tensorial Reconfiguration in Large Language Model Training [0.0]
  Context-Preservingial Reconfiguration (CPTR) enables dynamic complexity of weight tensors through structured factorization and adaptive contraction. Empirical evaluations demonstrate that CPTR improves coherence retention across extended sequences. Performance comparisons reveal that CPTR-enhanced models exhibit greater computational efficiency and reduced memory consumption.
  arXiv  Detail & Related papers  (2025-02-01T00:55:19Z)
• Towards Generalizable Trajectory Prediction Using Dual-Level Representation Learning And Adaptive Prompting [107.4034346788744]
  Existing vehicle trajectory prediction models struggle with generalizability, prediction uncertainties, and handling complex interactions. We propose Perceiver with Register queries (PerReg+), a novel trajectory prediction framework that introduces: (1) Dual-Level Representation Learning via Self-Distillation (SD) and Masked Reconstruction (MR), capturing global context and fine-grained details; (2) Enhanced Multimodality using register-based queries and pretraining, eliminating the need for clustering and suppression; and (3) Adaptive Prompt Tuning during fine-tuning, freezing the main architecture and optimizing a small number of prompts for efficient adaptation.
  arXiv  Detail & Related papers  (2025-01-08T20:11:09Z)
• An accuracy improving method for advertising click through rate prediction based on enhanced xDeepFM model [0.0]
  This paper proposes an improved CTR prediction model based on the xDeepFM architecture. By integrating a multi-head attention mechanism, the model can simultaneously focus on different aspects of feature interactions. Experimental results on the Criteo dataset demonstrate that the proposed model outperforms other state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-11-21T03:21:29Z)
• NeSHFS: Neighborhood Search with Heuristic-based Feature Selection for Click-Through Rate Prediction [1.3805049652130312]
  Click-through-rate (CTR) prediction plays an important role in online advertising and ad recommender systems. We propose a CTR algorithm named Neighborhood Search with Heuristic-based Feature Selection (NeSHFS) to enhance CTR prediction performance.
  arXiv  Detail & Related papers  (2024-09-13T10:43:18Z)
• ELASTIC: Efficient Linear Attention for Sequential Interest Compression [5.689306819772134]
  State-of-the-art sequential recommendation models heavily rely on transformer's attention mechanism. We propose ELASTIC, an Efficient Linear Attention for SequenTial Interest Compression. We conduct extensive experiments on various public datasets and compare it with several strong sequential recommenders.
  arXiv  Detail & Related papers  (2024-08-18T06:41:46Z)
• A Click-Through Rate Prediction Method Based on Cross-Importance of Multi-Order Features [4.820576346277399]
  This paper proposes a new model, FiiNet (Multiple Order Feature Interaction Importance Neural Networks) The model first uses the selective kernel network (SKNet) to explicitly construct multi-order feature crosses. It dynamically learns the importance of feature interaction combinations in a fine grained manner.
  arXiv  Detail & Related papers  (2024-05-14T16:05:57Z)
• Collaborative Filtering Based on Diffusion Models: Unveiling the Potential of High-Order Connectivity [10.683635786183894]
  CF-Diff is a new diffusion model-based collaborative filtering method. It is capable of making full use of collaborative signals along with multi-hop neighbors. It achieves remarkable gains up to 7.29% compared to the best competitor.
  arXiv  Detail & Related papers  (2024-04-22T14:49:46Z)
• Understanding Augmentation-based Self-Supervised Representation Learning via RKHS Approximation and Regression [53.15502562048627]
  Recent work has built the connection between self-supervised learning and the approximation of the top eigenspace of a graph Laplacian operator. This work delves into a statistical analysis of augmentation-based pretraining.
  arXiv  Detail & Related papers  (2023-06-01T15:18:55Z)
• DELTA: Dynamic Embedding Learning with Truncated Conscious Attention for CTR Prediction [61.68415731896613]
  Click-Through Rate (CTR) prediction is a pivotal task in product and content recommendation. We propose a model that enables Dynamic Embedding Learning with Truncated Conscious Attention for CTR prediction.
  arXiv  Detail & Related papers  (2023-05-03T12:34:45Z)
• IPCC-TP: Utilizing Incremental Pearson Correlation Coefficient for Joint Multi-Agent Trajectory Prediction [73.25645602768158]
  IPCC-TP is a novel relevance-aware module based on Incremental Pearson Correlation Coefficient to improve multi-agent interaction modeling. Our module can be conveniently embedded into existing multi-agent prediction methods to extend original motion distribution decoders.
  arXiv  Detail & Related papers  (2023-03-01T15:16:56Z)
• Directed Acyclic Graph Factorization Machines for CTR Prediction via Knowledge Distillation [65.62538699160085]
  We propose a Directed Acyclic Graph Factorization Machine (KD-DAGFM) to learn the high-order feature interactions from existing complex interaction models for CTR prediction via Knowledge Distillation. KD-DAGFM achieves the best performance with less than 21.5% FLOPs of the state-of-the-art method on both online and offline experiments.
  arXiv  Detail & Related papers  (2022-11-21T03:09:42Z)
• Contextual Squeeze-and-Excitation for Efficient Few-Shot Image Classification [57.36281142038042]
  We present a new adaptive block called Contextual Squeeze-and-Excitation (CaSE) that adjusts a pretrained neural network on a new task to significantly improve performance. We also present a new training protocol based on Coordinate-Descent called UpperCaSE that exploits meta-trained CaSE blocks and fine-tuning routines for efficient adaptation.
  arXiv  Detail & Related papers  (2022-06-20T15:25:08Z)
• CascadER: Cross-Modal Cascading for Knowledge Graph Link Prediction [22.96768147978534]
  We propose a tiered ranking architecture CascadER to maintain the ranking accuracy of full ensembling while improving efficiency considerably. CascadER uses LMs to rerank the outputs of more efficient base KGEs, relying on an adaptive subset selection scheme aimed at invoking the LMs minimally while maximizing accuracy gain over the KGE. Our empirical analyses reveal that diversity of models across modalities and preservation of individual models' confidence signals help explain the effectiveness of CascadER.
  arXiv  Detail & Related papers  (2022-05-16T22:55:45Z)
• Dynamic Parameterized Network for CTR Prediction [6.749659219776502]
  We proposed a novel plug-in operation, Dynamic ized Operation (DPO), to learn both explicit and implicit interaction instance-wisely. We showed that the introduction of DPO into DNN modules and Attention modules can respectively benefit two main tasks in click-through rate (CTR) prediction. Our Dynamic ized Networks significantly outperforms state-of-the-art methods in the offline experiments on the public dataset and real-world production dataset.
  arXiv  Detail & Related papers  (2021-11-09T08:15:03Z)
• CATRO: Channel Pruning via Class-Aware Trace Ratio Optimization [61.71504948770445]
  We propose a novel channel pruning method via Class-Aware Trace Ratio Optimization (CATRO) to reduce the computational burden and accelerate the model inference. We show that CATRO achieves higher accuracy with similar cost or lower cost with similar accuracy than other state-of-the-art channel pruning algorithms. Because of its class-aware property, CATRO is suitable to prune efficient networks adaptively for various classification subtasks, enhancing handy deployment and usage of deep networks in real-world applications.
  arXiv  Detail & Related papers  (2021-10-21T06:26:31Z)
• AdnFM: An Attentive DenseNet based Factorization Machine for CTR Prediction [11.958336595818267]
  We propose a novel model called Attentive DenseNet based Factorization Machines (AdnFM) AdnFM can extract more comprehensive deep features by using all the hidden layers from a feed-forward neural network as implicit high-order features. Experiments on two real-world datasets show that the proposed model can effectively improve the performance of Click-Through-Rate prediction.
  arXiv  Detail & Related papers  (2020-12-20T01:00:39Z)
• DAIS: Automatic Channel Pruning via Differentiable Annealing Indicator Search [55.164053971213576]
  convolutional neural network has achieved great success in fulfilling computer vision tasks despite large computation overhead. Structured (channel) pruning is usually applied to reduce the model redundancy while preserving the network structure. Existing structured pruning methods require hand-crafted rules which may lead to tremendous pruning space.
  arXiv  Detail & Related papers  (2020-11-04T07:43:01Z)
• Towards Automated Neural Interaction Discovery for Click-Through Rate Prediction [64.03526633651218]
  Click-Through Rate (CTR) prediction is one of the most important machine learning tasks in recommender systems. We propose an automated interaction architecture discovering framework for CTR prediction named AutoCTR.
  arXiv  Detail & Related papers  (2020-06-29T04:33:01Z)
• Feature Interaction based Neural Network for Click-Through Rate Prediction [5.095988654970358]
  We propose a Feature Interaction based Neural Network (FINN) which is able to model feature interaction via a 3-dimention relation tensor. We show that our deep FINN model outperforms other state-of-the-art deep models such as PNN and DeepFM. It also indicates that our models can effectively learn the feature interactions, and achieve better performances in real-world datasets.
  arXiv  Detail & Related papers  (2020-06-07T03:53:24Z)
• SAC: Accelerating and Structuring Self-Attention via Sparse Adaptive Connection [51.376723069962]
  We present a method for accelerating and structuring self-attentions: Sparse Adaptive Connection. In SAC, we regard the input sequence as a graph and attention operations are performed between linked nodes. We show that SAC is competitive with state-of-the-art models while significantly reducing memory cost.
  arXiv  Detail & Related papers  (2020-03-22T07:58:44Z)
• Cascaded Human-Object Interaction Recognition [175.60439054047043]
  We introduce a cascade architecture for a multi-stage, coarse-to-fine HOI understanding. At each stage, an instance localization network progressively refines HOI proposals and feeds them into an interaction recognition network. With our carefully-designed human-centric relation features, these two modules work collaboratively towards effective interaction understanding.
  arXiv  Detail & Related papers  (2020-03-09T17:05:04Z)
• Supervised Learning for Non-Sequential Data: A Canonical Polyadic Decomposition Approach [85.12934750565971]
  Efficient modelling of feature interactions underpins supervised learning for non-sequential tasks. To alleviate this issue, it has been proposed to implicitly represent the model parameters as a tensor. For enhanced expressiveness, we generalize the framework to allow feature mapping to arbitrarily high-dimensional feature vectors.
  arXiv  Detail & Related papers  (2020-01-27T22:38:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.