Memorize, Factorize, or be Na\"ive: Learning Optimal Feature Interaction Methods for CTR Prediction

• URL: http://arxiv.org/abs/2108.01265v1
• Date: Tue, 3 Aug 2021 03:03:34 GMT
• Title: Memorize, Factorize, or be Na\"ive: Learning Optimal Feature Interaction Methods for CTR Prediction
• Authors: Fuyuan Lyu, Xing Tang, Huifeng Guo, Ruiming Tang, Xiuqiang He, Rui Zhang, Xue Liu
• Abstract summary: We propose a framework called OptInter which finds the most suitable modelling method for each feature interaction. Our experiments show that OptInter improves the best performed state-of-the-art baseline deep CTR models by up to 2.21%.
• Score: 29.343267933348372
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Click-through rate prediction is one of the core tasks in commercial recommender systems. It aims to predict the probability of a user clicking a particular item given user and item features. As feature interactions bring in non-linearity, they are widely adopted to improve the performance of CTR prediction models. Therefore, effectively modelling feature interactions has attracted much attention in both the research and industry field. The current approaches can generally be categorized into three classes: (1) na\"ive methods, which do not model feature interactions and only use original features; (2) memorized methods, which memorize feature interactions by explicitly viewing them as new features and assigning trainable embeddings; (3) factorized methods, which learn latent vectors for original features and implicitly model feature interactions through factorization functions. Studies have shown that modelling feature interactions by one of these methods alone are suboptimal due to the unique characteristics of different feature interactions. To address this issue, we first propose a general framework called OptInter which finds the most suitable modelling method for each feature interaction. Different state-of-the-art deep CTR models can be viewed as instances of OptInter. To realize the functionality of OptInter, we also introduce a learning algorithm that automatically searches for the optimal modelling method. We conduct extensive experiments on four large datasets. Our experiments show that OptInter improves the best performed state-of-the-art baseline deep CTR models by up to 2.21%. Compared to the memorized method, which also outperforms baselines, we reduce up to 91% parameters. In addition, we conduct several ablation studies to investigate the influence of different components of OptInter. Finally, we provide interpretable discussions on the results of OptInter.

Related papers

• Exploring the cloud of feature interaction scores in a Rashomon set [17.775145325515993]
  We introduce the feature interaction score (FIS) in the context of a Rashomon set. We demonstrate the properties of the FIS via synthetic data and draw connections to other areas of statistics. Our results suggest that the proposed FIS can provide valuable insights into the nature of feature interactions in machine learning models.
  arXiv  Detail & Related papers  (2023-05-17T13:05:26Z)
• Coarse-to-Fine Knowledge-Enhanced Multi-Interest Learning Framework for Multi-Behavior Recommendation [52.89816309759537]
  Multi-types of behaviors (e.g., clicking, adding to cart, purchasing, etc.) widely exist in most real-world recommendation scenarios. The state-of-the-art multi-behavior models learn behavior dependencies indistinguishably with all historical interactions as input. We propose a novel Coarse-to-fine Knowledge-enhanced Multi-interest Learning framework to learn shared and behavior-specific interests for different behaviors.
  arXiv  Detail & Related papers  (2022-08-03T05:28:14Z)
• Meta-Wrapper: Differentiable Wrapping Operator for User Interest Selection in CTR Prediction [97.99938802797377]
  Click-through rate (CTR) prediction, whose goal is to predict the probability of the user to click on an item, has become increasingly significant in recommender systems. Recent deep learning models with the ability to automatically extract the user interest from his/her behaviors have achieved great success. We propose a novel approach under the framework of the wrapper method, which is named Meta-Wrapper.
  arXiv  Detail & Related papers  (2022-06-28T03:28:15Z)
• Masked Transformer for Neighhourhood-aware Click-Through Rate Prediction [74.52904110197004]
  We propose Neighbor-Interaction based CTR prediction, which put this task into a Heterogeneous Information Network (HIN) setting. In order to enhance the representation of the local neighbourhood, we consider four types of topological interaction among the nodes. We conduct comprehensive experiments on two real world datasets and the experimental results show that our proposed method outperforms state-of-the-art CTR models significantly.
  arXiv  Detail & Related papers  (2022-01-25T12:44:23Z)
• Efficient Data-specific Model Search for Collaborative Filtering [56.60519991956558]
  Collaborative filtering (CF) is a fundamental approach for recommender systems. In this paper, motivated by the recent advances in automated machine learning (AutoML), we propose to design a data-specific CF model. Key here is a new framework that unifies state-of-the-art (SOTA) CF methods and splits them into disjoint stages of input encoding, embedding function, interaction and prediction function.
  arXiv  Detail & Related papers  (2021-06-14T14:30:32Z)
• Neural Graph Matching based Collaborative Filtering [13.086302251856756]
  We identify two different types of attribute interactions, inner and cross interactions. Existing models do not distinguish these two types of attribute interactions. We propose a neural Graph Matching based Collaborative Filtering model (GMCF) Our model outperforms state-of-the-art models.
  arXiv  Detail & Related papers  (2021-05-10T01:51:46Z)
• AutoDis: Automatic Discretization for Embedding Numerical Features in CTR Prediction [45.69943728028556]
  Learning sophisticated feature interactions is crucial for Click-Through Rate (CTR) prediction in recommender systems. Various deep CTR models follow an Embedding & Feature Interaction paradigm. We propose AutoDis, a framework that discretizes features in numerical fields automatically and is optimized with CTR models in an end-to-end manner.
  arXiv  Detail & Related papers  (2020-12-16T14:31:31Z)
• CAN: Feature Co-Action for Click-Through Rate Prediction [42.251405364218805]
  We propose a Co-Action Network (CAN) to approximate the explicit pairwise feature interactions. CAN outperforms state-of-the-art CTR models and the cartesian product method. CAN has been deployed in the display advertisement system in Alibaba, obtaining 12% improvement on CTR.
  arXiv  Detail & Related papers  (2020-11-11T08:33:07Z)
• FIVES: Feature Interaction Via Edge Search for Large-Scale Tabular Data [106.76845921324704]
  We propose a novel method named Feature Interaction Via Edge Search (FIVES) FIVES formulates the task of interactive feature generation as searching for edges on the defined feature graph. In this paper, we present our theoretical evidence that motivates us to search for useful interactive features with increasing order.
  arXiv  Detail & Related papers  (2020-07-29T03:33:18Z)
• AutoFIS: Automatic Feature Interaction Selection in Factorization Models for Click-Through Rate Prediction [75.16836697734995]
  We propose a two-stage algorithm called Automatic Feature Interaction Selection (AutoFIS) AutoFIS can automatically identify important feature interactions for factorization models with computational cost just equivalent to training the target model to convergence. AutoFIS has been deployed onto the training platform of Huawei App Store recommendation service.
  arXiv  Detail & Related papers  (2020-03-25T06:53:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.