SuperCone: Modeling Heterogeneous Experts with Concept Meta-learning for Unified Predictive Segments System

• URL: http://arxiv.org/abs/2203.07029v1
• Date: Wed, 9 Mar 2022 04:11:39 GMT
• Title: SuperCone: Modeling Heterogeneous Experts with Concept Meta-learning for Unified Predictive Segments System
• Authors: Keqian Li, Yifan Hu
• Abstract summary: We present SuperCone, our unified predicative segments system. It builds on top of a flat concept representation that summarizes each user's heterogeneous digital footprints. It can outperform state-of-the-art recommendation and ranking algorithms on a wide range of predicative segment tasks.
• Score: 8.917697023052257
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Understanding users through predicative segments play an essential role for modern enterprises for more efficient and efficient information exchange. For example, by predicting whether a user has particular interest in a particular area of sports or entertainment, we can better serve the user with more relevant and tailored content. However, there exists a large number of long tail prediction tasks that are hard to capture by off the shelf model architectures due to data scarcity and task heterogeneity. In this work, we present SuperCone, our unified predicative segments system that addresses the above challenges. It builds on top of a flat concept representation that summarizes each user's heterogeneous digital footprints, and uniformly models each of the prediction task using an approach called "super learning ", that is, combining prediction models with diverse architectures or learning method that are not compatible with each other or even completely unknown. Following this, we provide end to end deep learning architecture design that flexibly learns to attend to best suited heterogeneous experts while at the same time learns deep representations of the input concepts that augments the above experts by capturing unique signal. Experiments show that SuperCone can outperform state-of-the-art recommendation and ranking algorithms on a wide range of predicative segment tasks, as well as several public structured data learning benchmarks.

Related papers

• LLM-assisted Explicit and Implicit Multi-interest Learning Framework for Sequential Recommendation [50.98046887582194]
  We propose an explicit and implicit multi-interest learning framework to model user interests on two levels: behavior and semantics. The proposed EIMF framework effectively and efficiently combines small models with LLM to improve the accuracy of multi-interest modeling.
  arXiv  Detail & Related papers  (2024-11-14T13:00:23Z)
• AMEND: A Mixture of Experts Framework for Long-tailed Trajectory Prediction [6.724750970258851]
  We propose a modular model-agnostic framework for trajectory prediction. Each expert is trained with a specialized skill with respect to a particular part of the data. To produce predictions, we utilise a router network that selects the best expert by generating relative confidence scores.
  arXiv  Detail & Related papers  (2024-02-13T02:43:41Z)
• Understanding Before Recommendation: Semantic Aspect-Aware Review Exploitation via Large Language Models [53.337728969143086]
  Recommendation systems harness user-item interactions like clicks and reviews to learn their representations. Previous studies improve recommendation accuracy and interpretability by modeling user preferences across various aspects and intents. We introduce a chain-based prompting approach to uncover semantic aspect-aware interactions.
  arXiv  Detail & Related papers  (2023-12-26T15:44:09Z)
• Fantastic Gains and Where to Find Them: On the Existence and Prospect of General Knowledge Transfer between Any Pretrained Model [74.62272538148245]
  We show that for arbitrary pairings of pretrained models, one model extracts significant data context unavailable in the other. We investigate if it is possible to transfer such "complementary" knowledge from one model to another without performance degradation.
  arXiv  Detail & Related papers  (2023-10-26T17:59:46Z)
• FedBone: Towards Large-Scale Federated Multi-Task Learning [13.835972363413884]
  In real-world applications, visual and natural language tasks typically require large-scale models to extract high-level abstract features. Existing HFML methods disregard the impact of gradient conflicts on multi-task optimization. We propose an innovative framework called FedBone, which enables the construction of large-scale models with better generalization.
  arXiv  Detail & Related papers  (2023-06-30T08:19:38Z)
• Personalized Federated Learning with Feature Alignment and Classifier Collaboration [13.320381377599245]
  Data heterogeneity is one of the most challenging issues in federated learning. One such approach in deep neural networks based tasks is employing a shared feature representation and learning a customized classifier head for each client. In this work, we conduct explicit local-global feature alignment by leveraging global semantic knowledge for learning a better representation.
  arXiv  Detail & Related papers  (2023-06-20T19:58:58Z)
• Deep networks for system identification: a Survey [56.34005280792013]
  System identification learns mathematical descriptions of dynamic systems from input-output data. Main aim of the identified model is to predict new data from previous observations. We discuss architectures commonly adopted in the literature, like feedforward, convolutional, and recurrent networks.
  arXiv  Detail & Related papers  (2023-01-30T12:38:31Z)
• Diversified Dynamic Routing for Vision Tasks [36.199659460868496]
  We propose a novel architecture where each layer is composed of a set of experts. In our method, the model is explicitly trained to solve the challenge of finding relevant partitioning of the data. We conduct several experiments on semantic segmentation on Cityscapes and object detection and instance segmentation on MS-COCO.
  arXiv  Detail & Related papers  (2022-09-26T23:27:51Z)
• A Review of Sparse Expert Models in Deep Learning [23.721204843236006]
  Sparse expert models are a thirty-year old concept re-emerging as a popular architecture in deep learning. We review the concept of sparse expert models, provide a basic description of the common algorithms, and contextualize the advances in the deep learning era.
  arXiv  Detail & Related papers  (2022-09-04T18:00:29Z)
• Revisiting Contrastive Methods for Unsupervised Learning of Visual Representations [78.12377360145078]
  Contrastive self-supervised learning has outperformed supervised pretraining on many downstream tasks like segmentation and object detection. In this paper, we first study how biases in the dataset affect existing methods. We show that current contrastive approaches work surprisingly well across: (i) object- versus scene-centric, (ii) uniform versus long-tailed and (iii) general versus domain-specific datasets.
  arXiv  Detail & Related papers  (2021-06-10T17:59:13Z)
• Exploiting Shared Representations for Personalized Federated Learning [54.65133770989836]
  We propose a novel federated learning framework and algorithm for learning a shared data representation across clients and unique local heads for each client. Our algorithm harnesses the distributed computational power across clients to perform many local-updates with respect to the low-dimensional local parameters for every update of the representation. This result is of interest beyond federated learning to a broad class of problems in which we aim to learn a shared low-dimensional representation among data distributions.
  arXiv  Detail & Related papers  (2021-02-14T05:36:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.