Towards Unifying Feature Interaction Models for Click-Through Rate Prediction

• URL: http://arxiv.org/abs/2411.12441v1
• Date: Tue, 19 Nov 2024 12:04:02 GMT
• Title: Towards Unifying Feature Interaction Models for Click-Through Rate Prediction
• Authors: Yu Kang, Junwei Pan, Jipeng Jin, Shudong Huang, Xiaofeng Gao, Lei Xiao,
• Abstract summary: We propose a general framework called IPA to unify existing models. We demonstrate that most existing models can be categorized within our framework by making specific choices for these three components. We introduce a novel model that achieves competitive results compared to state-of-the-art CTR models.
• Score: 19.149554121852724
• License:
• Abstract: Modeling feature interactions plays a crucial role in accurately predicting click-through rates (CTR) in advertising systems. To capture the intricate patterns of interaction, many existing models employ matrix-factorization techniques to represent features as lower-dimensional embedding vectors, enabling the modeling of interactions as products between these embeddings. In this paper, we propose a general framework called IPA to systematically unify these models. Our framework comprises three key components: the Interaction Function, which facilitates feature interaction; the Layer Pooling, which constructs higher-level interaction layers; and the Layer Aggregator, which combines the outputs of all layers to serve as input for the subsequent classifier. We demonstrate that most existing models can be categorized within our framework by making specific choices for these three components. Through extensive experiments and a dimensional collapse analysis, we evaluate the performance of these choices. Furthermore, by leveraging the most powerful components within our framework, we introduce a novel model that achieves competitive results compared to state-of-the-art CTR models. PFL gets significant GMV lift during online A/B test in Tencent's advertising platform and has been deployed as the production model in several primary scenarios.

Related papers

• A Collaborative Ensemble Framework for CTR Prediction [73.59868761656317]
  We propose a novel framework, Collaborative Ensemble Training Network (CETNet), to leverage multiple distinct models. Unlike naive model scaling, our approach emphasizes diversity and collaboration through collaborative learning. We validate our framework on three public datasets and a large-scale industrial dataset from Meta.
  arXiv  Detail & Related papers  (2024-11-20T20:38:56Z)
• Feature Interaction Fusion Self-Distillation Network For CTR Prediction [14.12775753361368]
  Click-Through Rate (CTR) prediction plays a vital role in recommender systems, online advertising, and search engines. We propose FSDNet, a CTR prediction framework incorporating a plug-and-play fusion self-distillation module.
  arXiv  Detail & Related papers  (2024-11-12T03:05:03Z)
• Interfacing Foundation Models' Embeddings [131.0352288172788]
  We present FIND, a generalized interface for aligning foundation models' embeddings with unified image and dataset-level understanding spanning modality and granularity. In light of the interleaved embedding space, we introduce FIND-Bench, which introduces new training and evaluation annotations to the COCO dataset for interleaved segmentation and retrieval.
  arXiv  Detail & Related papers  (2023-12-12T18:58:02Z)
• Part-guided Relational Transformers for Fine-grained Visual Recognition [59.20531172172135]
  We propose a framework to learn the discriminative part features and explore correlations with a feature transformation module. Our proposed approach does not rely on additional part branches and reaches state-the-of-art performance on 3-of-the-level object recognition.
  arXiv  Detail & Related papers  (2022-12-28T03:45:56Z)
• Colar: Effective and Efficient Online Action Detection by Consulting Exemplars [102.28515426925621]
  We develop an effective exemplar-consultation mechanism that first measures the similarity between a frame and exemplary frames, and then aggregates exemplary features based on the similarity weights. Due to the complementarity from the category-level modeling, our method employs a lightweight architecture but achieves new high performance on three benchmarks.
  arXiv  Detail & Related papers  (2022-03-02T12:13:08Z)
• Masked Transformer for Neighhourhood-aware Click-Through Rate Prediction [74.52904110197004]
  We propose Neighbor-Interaction based CTR prediction, which put this task into a Heterogeneous Information Network (HIN) setting. In order to enhance the representation of the local neighbourhood, we consider four types of topological interaction among the nodes. We conduct comprehensive experiments on two real world datasets and the experimental results show that our proposed method outperforms state-of-the-art CTR models significantly.
  arXiv  Detail & Related papers  (2022-01-25T12:44:23Z)
• Multiscale Generative Models: Improving Performance of a Generative Model Using Feedback from Other Dependent Generative Models [10.053377705165786]
  We take a first step towards building interacting generative models (GANs) that reflects the interaction in real world. We build and analyze a hierarchical set-up where a higher-level GAN is conditioned on the output of multiple lower-level GANs. We present a technique of using feedback from the higher-level GAN to improve performance of lower-level GANs.
  arXiv  Detail & Related papers  (2022-01-24T13:05:56Z)
• Memorize, Factorize, or be Na\"ive: Learning Optimal Feature Interaction Methods for CTR Prediction [29.343267933348372]
  We propose a framework called OptInter which finds the most suitable modelling method for each feature interaction. Our experiments show that OptInter improves the best performed state-of-the-art baseline deep CTR models by up to 2.21%.
  arXiv  Detail & Related papers  (2021-08-03T03:03:34Z)
• Visual Composite Set Detection Using Part-and-Sum Transformers [74.26037922682355]
  We present a new approach, denoted Part-and-Sum detection Transformer (PST), to perform end-to-end composite set detection. PST achieves state-of-the-art results among single-stage models, while nearly matching the results of custom-designed two-stage models.
  arXiv  Detail & Related papers  (2021-05-05T16:31:32Z)
• AutoDis: Automatic Discretization for Embedding Numerical Features in CTR Prediction [45.69943728028556]
  Learning sophisticated feature interactions is crucial for Click-Through Rate (CTR) prediction in recommender systems. Various deep CTR models follow an Embedding & Feature Interaction paradigm. We propose AutoDis, a framework that discretizes features in numerical fields automatically and is optimized with CTR models in an end-to-end manner.
  arXiv  Detail & Related papers  (2020-12-16T14:31:31Z)
• Feature Interaction based Neural Network for Click-Through Rate Prediction [5.095988654970358]
  We propose a Feature Interaction based Neural Network (FINN) which is able to model feature interaction via a 3-dimention relation tensor. We show that our deep FINN model outperforms other state-of-the-art deep models such as PNN and DeepFM. It also indicates that our models can effectively learn the feature interactions, and achieve better performances in real-world datasets.
  arXiv  Detail & Related papers  (2020-06-07T03:53:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.