SuperCone: Modeling Heterogeneous Experts with Concept Meta-learning for Unified Predictive Segments System

• URL: http://arxiv.org/abs/2203.07029v1
• Date: Wed, 9 Mar 2022 04:11:39 GMT
• Title: SuperCone: Modeling Heterogeneous Experts with Concept Meta-learning for Unified Predictive Segments System
• Authors: Keqian Li, Yifan Hu
• Abstract summary: We present SuperCone, our unified predicative segments system. It builds on top of a flat concept representation that summarizes each user's heterogeneous digital footprints. It can outperform state-of-the-art recommendation and ranking algorithms on a wide range of predicative segment tasks.
• Score: 8.917697023052257
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Understanding users through predicative segments play an essential role for modern enterprises for more efficient and efficient information exchange. For example, by predicting whether a user has particular interest in a particular area of sports or entertainment, we can better serve the user with more relevant and tailored content. However, there exists a large number of long tail prediction tasks that are hard to capture by off the shelf model architectures due to data scarcity and task heterogeneity. In this work, we present SuperCone, our unified predicative segments system that addresses the above challenges. It builds on top of a flat concept representation that summarizes each user's heterogeneous digital footprints, and uniformly models each of the prediction task using an approach called "super learning ", that is, combining prediction models with diverse architectures or learning method that are not compatible with each other or even completely unknown. Following this, we provide end to end deep learning architecture design that flexibly learns to attend to best suited heterogeneous experts while at the same time learns deep representations of the input concepts that augments the above experts by capturing unique signal. Experiments show that SuperCone can outperform state-of-the-art recommendation and ranking algorithms on a wide range of predicative segment tasks, as well as several public structured data learning benchmarks.

Related papers

• AMEND: A Mixture of Experts Framework for Long-tailed Trajectory Prediction [6.724750970258851]
  We propose a modular model-agnostic framework for trajectory prediction. Each expert is trained with a specialized skill with respect to a particular part of the data. To produce predictions, we utilise a router network that selects the best expert by generating relative confidence scores.
  arXiv  Detail & Related papers  (2024-02-13T02:43:41Z)
• Fantastic Gains and Where to Find Them: On the Existence and Prospect of General Knowledge Transfer between Any Pretrained Model [74.62272538148245]
  We show that for arbitrary pairings of pretrained models, one model extracts significant data context unavailable in the other. We investigate if it is possible to transfer such "complementary" knowledge from one model to another without performance degradation.
  arXiv  Detail & Related papers  (2023-10-26T17:59:46Z)
• FedBone: Towards Large-Scale Federated Multi-Task Learning [13.835972363413884]
  In real-world applications, visual and natural language tasks typically require large-scale models to extract high-level abstract features. Existing HFML methods disregard the impact of gradient conflicts on multi-task optimization. We propose an innovative framework called FedBone, which enables the construction of large-scale models with better generalization.
  arXiv  Detail & Related papers  (2023-06-30T08:19:38Z)
• Personalized Federated Learning with Feature Alignment and Classifier Collaboration [13.320381377599245]
  Data heterogeneity is one of the most challenging issues in federated learning. One such approach in deep neural networks based tasks is employing a shared feature representation and learning a customized classifier head for each client. In this work, we conduct explicit local-global feature alignment by leveraging global semantic knowledge for learning a better representation.
  arXiv  Detail & Related papers  (2023-06-20T19:58:58Z)
• Deep networks for system identification: a Survey [56.34005280792013]
  System identification learns mathematical descriptions of dynamic systems from input-output data. Main aim of the identified model is to predict new data from previous observations. We discuss architectures commonly adopted in the literature, like feedforward, convolutional, and recurrent networks.
  arXiv  Detail & Related papers  (2023-01-30T12:38:31Z)
• Diversified Dynamic Routing for Vision Tasks [36.199659460868496]
  We propose a novel architecture where each layer is composed of a set of experts. In our method, the model is explicitly trained to solve the challenge of finding relevant partitioning of the data. We conduct several experiments on semantic segmentation on Cityscapes and object detection and instance segmentation on MS-COCO.
  arXiv  Detail & Related papers  (2022-09-26T23:27:51Z)
• A Review of Sparse Expert Models in Deep Learning [23.721204843236006]
  Sparse expert models are a thirty-year old concept re-emerging as a popular architecture in deep learning. We review the concept of sparse expert models, provide a basic description of the common algorithms, and contextualize the advances in the deep learning era.
  arXiv  Detail & Related papers  (2022-09-04T18:00:29Z)
• HyperImpute: Generalized Iterative Imputation with Automatic Model Selection [77.86861638371926]
  We propose a generalized iterative imputation framework for adaptively and automatically configuring column-wise models. We provide a concrete implementation with out-of-the-box learners, simulators, and interfaces.
  arXiv  Detail & Related papers  (2022-06-15T19:10:35Z)
• Self-Supervised Visual Representation Learning with Semantic Grouping [50.14703605659837]
  We tackle the problem of learning visual representations from unlabeled scene-centric data. We propose contrastive learning from data-driven semantic slots, namely SlotCon, for joint semantic grouping and representation learning.
  arXiv  Detail & Related papers  (2022-05-30T17:50:59Z)
• Revisiting Contrastive Methods for Unsupervised Learning of Visual Representations [78.12377360145078]
  Contrastive self-supervised learning has outperformed supervised pretraining on many downstream tasks like segmentation and object detection. In this paper, we first study how biases in the dataset affect existing methods. We show that current contrastive approaches work surprisingly well across: (i) object- versus scene-centric, (ii) uniform versus long-tailed and (iii) general versus domain-specific datasets.
  arXiv  Detail & Related papers  (2021-06-10T17:59:13Z)
• Exploiting Shared Representations for Personalized Federated Learning [54.65133770989836]
  We propose a novel federated learning framework and algorithm for learning a shared data representation across clients and unique local heads for each client. Our algorithm harnesses the distributed computational power across clients to perform many local-updates with respect to the low-dimensional local parameters for every update of the representation. This result is of interest beyond federated learning to a broad class of problems in which we aim to learn a shared low-dimensional representation among data distributions.
  arXiv  Detail & Related papers  (2021-02-14T05:36:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.