HMDN: Hierarchical Multi-Distribution Network for Click-Through Rate Prediction

• URL: http://arxiv.org/abs/2408.01332v1
• Date: Fri, 2 Aug 2024 15:29:59 GMT
• Title: HMDN: Hierarchical Multi-Distribution Network for Click-Through Rate Prediction
• Authors: Xingyu Lou, Yu Yang, Kuiyao Dong, Heyuan Huang, Wenyi Yu, Ping Wang, Xiu Li, Jun Wang,
• Abstract summary: We propose a flexible modeling paradigm, named Hierarchical Multi-Distribution Network (HMDN) HMDN efficiently models mixed multi-distributions and can seamlessly integrate with existing multi-distribution methods. Experimental results on both public and industrial datasets validate the effectiveness and flexibility of HMDN.
• Score: 26.32695178700689
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As the recommendation service needs to address increasingly diverse distributions, such as multi-population, multi-scenario, multitarget, and multi-interest, more and more recent works have focused on multi-distribution modeling and achieved great progress. However, most of them only consider modeling in a single multi-distribution manner, ignoring that mixed multi-distributions often coexist and form hierarchical relationships. To address these challenges, we propose a flexible modeling paradigm, named Hierarchical Multi-Distribution Network (HMDN), which efficiently models these hierarchical relationships and can seamlessly integrate with existing multi-distribution methods, such as Mixture of-Experts (MoE) and Dynamic-Weight (DW) models. Specifically, we first design a hierarchical multi-distribution representation refinement module, employing a multi-level residual quantization to obtain fine-grained hierarchical representation. Then, the refined hierarchical representation is integrated into the existing single multi-distribution models, seamlessly expanding them into mixed multi-distribution models. Experimental results on both public and industrial datasets validate the effectiveness and flexibility of HMDN.

Related papers

• Learning Multimodal Latent Generative Models with Energy-Based Prior [3.6648642834198797]
  We propose a novel framework that integrates the latent generative model with the EBM. This approach results in a more expressive and informative prior, better-capturing of information across multiple modalities.
  arXiv  Detail & Related papers  (2024-09-30T01:38:26Z)
• Diffusion Models For Multi-Modal Generative Modeling [32.61765315067488]
  We propose a principled way to define a diffusion model by constructing a unified multi-modal diffusion model in a common diffusion space. We propose several multimodal generation settings to verify our framework, including image transition, masked-image training, joint image-label and joint image-representation generative modeling.
  arXiv  Detail & Related papers  (2024-07-24T18:04:17Z)
• U3M: Unbiased Multiscale Modal Fusion Model for Multimodal Semantic Segmentation [63.31007867379312]
  We introduce U3M: An Unbiased Multiscale Modal Fusion Model for Multimodal Semantics. We employ feature fusion at multiple scales to ensure the effective extraction and integration of both global and local features. Experimental results demonstrate that our approach achieves superior performance across multiple datasets.
  arXiv  Detail & Related papers  (2024-05-24T08:58:48Z)
• Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts [54.529880848937104]
  We develop a unified MLLM with the MoE architecture, named Uni-MoE, that can handle a wide array of modalities. Specifically, it features modality-specific encoders with connectors for a unified multimodal representation. We evaluate the instruction-tuned Uni-MoE on a comprehensive set of multimodal datasets.
  arXiv  Detail & Related papers  (2024-05-18T12:16:01Z)
• Toward Robust Multimodal Learning using Multimodal Foundational Models [30.755818450393637]
  We propose TRML, Toward Robust Multimodal Learning using Multimodal Foundational Models. TRML employs generated virtual modalities to replace missing modalities. We also design a semantic matching learning module to align semantic spaces generated and missing modalities.
  arXiv  Detail & Related papers  (2024-01-20T04:46:43Z)
• Efficient Multimodal Diffusion Models Using Joint Data Infilling with Partially Shared U-Net [20.437172251393257]
  Partially Shared U-Net (PS-U-Net) is an efficient multimodal diffusion model that allows text and image inputs to pass through dedicated layers and skip-connections for preserving modality-specific fine-grained details. Inspired by image inpainting, we also propose a new efficient multimodal sampling method that introduces new scenarios for conditional generation while only requiring a simple joint distribution to be learned. Our empirical exploration of the MS-COCO dataset demonstrates that our method generates multimodal text and image data with higher quality compared to existing multimodal diffusion models.
  arXiv  Detail & Related papers  (2023-11-28T04:34:44Z)
• Unified Multi-modal Unsupervised Representation Learning for Skeleton-based Action Understanding [62.70450216120704]
  Unsupervised pre-training has shown great success in skeleton-based action understanding. We propose a Unified Multimodal Unsupervised Representation Learning framework, called UmURL. UmURL exploits an efficient early-fusion strategy to jointly encode the multi-modal features in a single-stream manner.
  arXiv  Detail & Related papers  (2023-11-06T13:56:57Z)
• Learning Hierarchical Features with Joint Latent Space Energy-Based Prior [44.4434704520236]
  We study the fundamental problem of multi-layer generator models in learning hierarchical representations. We propose a joint latent space EBM prior model with multi-layer latent variables for effective hierarchical representation learning.
  arXiv  Detail & Related papers  (2023-10-14T15:44:14Z)
• Multi-scale Cooperative Multimodal Transformers for Multimodal Sentiment Analysis in Videos [58.93586436289648]
  We propose a multi-scale cooperative multimodal transformer (MCMulT) architecture for multimodal sentiment analysis. Our model outperforms existing approaches on unaligned multimodal sequences and has strong performance on aligned multimodal sequences.
  arXiv  Detail & Related papers  (2022-06-16T07:47:57Z)
• Relating by Contrasting: A Data-efficient Framework for Multimodal Generative Models [86.9292779620645]
  We develop a contrastive framework for generative model learning, allowing us to train the model not just by the commonality between modalities, but by the distinction between "related" and "unrelated" multimodal data. Under our proposed framework, the generative model can accurately identify related samples from unrelated ones, making it possible to make use of the plentiful unlabeled, unpaired multimodal data.
  arXiv  Detail & Related papers  (2020-07-02T15:08:11Z)
• Unsupervised multi-modal Styled Content Generation [61.040392094140245]
  UMMGAN is a novel architecture designed to better model multi-modal distributions in an unsupervised fashion. We show that UMMGAN effectively disentangles between modes and style, thereby providing an independent degree of control over the generated content.
  arXiv  Detail & Related papers  (2020-01-10T19:36:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.