Fusion Matters: Learning Fusion in Deep Click-through Rate Prediction Models

• URL: http://arxiv.org/abs/2411.15731v1
• Date: Sun, 24 Nov 2024 06:21:59 GMT
• Title: Fusion Matters: Learning Fusion in Deep Click-through Rate Prediction Models
• Authors: Kexin Zhang, Fuyuan Lyu, Xing Tang, Dugang Liu, Chen Ma, Kaize Ding, Xiuqiang He, Xue Liu,
• Abstract summary: We introduce OptFusion, a method that automates the learning of fusion, encompassing both the connection learning and the operation selection. Our experiments are conducted over three large-scale datasets.
• Score: 27.477136474888564
• License:
• Abstract: The evolution of previous Click-Through Rate (CTR) models has mainly been driven by proposing complex components, whether shallow or deep, that are adept at modeling feature interactions. However, there has been less focus on improving fusion design. Instead, two naive solutions, stacked and parallel fusion, are commonly used. Both solutions rely on pre-determined fusion connections and fixed fusion operations. It has been repetitively observed that changes in fusion design may result in different performances, highlighting the critical role that fusion plays in CTR models. While there have been attempts to refine these basic fusion strategies, these efforts have often been constrained to specific settings or dependent on specific components. Neural architecture search has also been introduced to partially deal with fusion design, but it comes with limitations. The complexity of the search space can lead to inefficient and ineffective results. To bridge this gap, we introduce OptFusion, a method that automates the learning of fusion, encompassing both the connection learning and the operation selection. We have proposed a one-shot learning algorithm tackling these tasks concurrently. Our experiments are conducted over three large-scale datasets. Extensive experiments prove both the effectiveness and efficiency of OptFusion in improving CTR model performance. Our code implementation is available here\url{https://github.com/kexin-kxzhang/OptFusion}.

Related papers

• Wolf2Pack: The AutoFusion Framework for Dynamic Parameter Fusion [4.164728134421114]
  We introduce AutoFusion, a framework that fuses distinct model parameters for multi-task learning without pre-trained checkpoints. We validate AutoFusion's effectiveness through experiments on commonly used benchmark datasets. Our framework offers a scalable and flexible solution for model integration, positioning it as a powerful tool for future research and practical applications.
  arXiv  Detail & Related papers  (2024-10-08T07:21:24Z)
• FusionBench: A Comprehensive Benchmark of Deep Model Fusion [78.80920533793595]
  Deep model fusion is a technique that unifies the predictions or parameters of several deep neural networks into a single model. FusionBench is the first comprehensive benchmark dedicated to deep model fusion.
  arXiv  Detail & Related papers  (2024-06-05T13:54:28Z)
• Fusion-Mamba for Cross-modality Object Detection [63.56296480951342]
  Cross-modality fusing information from different modalities effectively improves object detection performance. We design a Fusion-Mamba block (FMB) to map cross-modal features into a hidden state space for interaction. Our proposed approach outperforms the state-of-the-art methods on $m$AP with 5.9% on $M3FD$ and 4.9% on FLIR-Aligned datasets.
  arXiv  Detail & Related papers  (2024-04-14T05:28:46Z)
• Transformer Fusion with Optimal Transport [25.022849817421964]
  Fusion is a technique for merging multiple independently-trained neural networks in order to combine their capabilities. This paper presents a systematic approach for fusing two or more transformer-based networks exploiting Optimal Transport to (soft-)align the various architectural components.
  arXiv  Detail & Related papers  (2023-10-09T13:40:31Z)
• Deep Model Fusion: A Survey [37.39100741978586]
  Deep model fusion/merging is an emerging technique that merges the parameters or predictions of multiple deep learning models into a single one. It faces several challenges, including high computational cost, high-dimensional parameter space, interference between different heterogeneous models, etc.
  arXiv  Detail & Related papers  (2023-09-27T14:40:12Z)
• A Task-guided, Implicitly-searched and Meta-initialized Deep Model for Image Fusion [69.10255211811007]
  We present a Task-guided, Implicit-searched and Meta- generalizationd (TIM) deep model to address the image fusion problem in a challenging real-world scenario. Specifically, we propose a constrained strategy to incorporate information from downstream tasks to guide the unsupervised learning process of image fusion. Within this framework, we then design an implicit search scheme to automatically discover compact architectures for our fusion model with high efficiency.
  arXiv  Detail & Related papers  (2023-05-25T08:54:08Z)
• LRRNet: A Novel Representation Learning Guided Fusion Network for Infrared and Visible Images [98.36300655482196]
  We formulate the fusion task mathematically, and establish a connection between its optimal solution and the network architecture that can implement it. In particular we adopt a learnable representation approach to the fusion task, in which the construction of the fusion network architecture is guided by the optimisation algorithm producing the learnable model. Based on this novel network architecture, an end-to-end lightweight fusion network is constructed to fuse infrared and visible light images.
  arXiv  Detail & Related papers  (2023-04-11T12:11:23Z)
• Cooperation Learning Enhanced Colonic Polyp Segmentation Based on Transformer-CNN Fusion [21.6402447417878]
  We propose a hybrid network called Fusion-Transformer-HardNetMSEG (i.e., Fu-TransHNet) in this study. Fu-TransHNet uses deep learning of different mechanisms to fuse each other and is enhanced with multi-view collaborative learning techniques. Experimental results showed that the Fu-TransHNet network was superior to the existing methods on five widely used benchmark datasets.
  arXiv  Detail & Related papers  (2023-01-17T13:58:17Z)
• Light Field Saliency Detection with Dual Local Graph Learning andReciprocative Guidance [148.9832328803202]
  We model the infor-mation fusion within focal stack via graph networks. We build a novel dual graph modelto guide the focal stack fusion process using all-focus pat-terns.
  arXiv  Detail & Related papers  (2021-10-02T00:54:39Z)
• Image Fusion Transformer [75.71025138448287]
  In image fusion, images obtained from different sensors are fused to generate a single image with enhanced information. In recent years, state-of-the-art methods have adopted Convolution Neural Networks (CNNs) to encode meaningful features for image fusion. We propose a novel Image Fusion Transformer (IFT) where we develop a transformer-based multi-scale fusion strategy.
  arXiv  Detail & Related papers  (2021-07-19T16:42:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.