Mosaic: A Fleet of User Embedding Specialists for Recommendation at Meta

• URL: http://arxiv.org/abs/2607.24015v1
• Date: Mon, 27 Jul 2026 05:25:50 GMT
• Title: Mosaic: A Fleet of User Embedding Specialists for Recommendation at Meta
• Abstract summary: We present Mosaic, a user modeling platform that employs a fleet of specialists to learn user embeddings.<n>We also introduce CoEval and User Tower Zero-Out, new logging-free embedding evaluation framework.<n>Mosaic delivers consistent and significant offline NE improvements in addition to online gains.
• Score: 12.699812020588974
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: User representation is one of the highest-leverage modeling problems in industrial recommendation systems: a single advancement in how users are encoded can propagate across retrieval, ranking, and integrity tasks at platform scale. Prior industrial user representation work builds either a single user model that emits one or more embedding vectors or a shared backbone with task-specific adaptation. In this paper, we present Mosaic, a foundational user modeling platform that employs a fleet of specialists to learn user embeddings. The fleet comprises four architecturally diverse model families - memorization-driven, dense-heavy, sequential-based, and CoTrain models - each focusing on a distinct facet of user behavior. We developed MRM (Multi-task Relations Mining) and CRL (Cosine Redundancy Loss) techniques to maximize the marginal information contribution of each new specialist. We also introduce CoEval and User Tower Zero-Out, new logging-free embedding evaluation framework that improves development velocity while preserving downstream-aligned accuracy. Our hybrid CPU/GPU, online-and-offline serving stack allows each specialist to choose the adequate serving strategy to meet the freshness, latency, and computational requirements. Mosaic delivers consistent and significant offline NE improvements in addition to online gains.

Related papers

• m3BERT: A Modern, Multi-lingual, Matryoshka Bidirectional Encoder [50.2753694138149]
  We introduce m3BERT: a Modern, Multi-lingual, Matryoshka Bidirectional representations.<n>It features a novel pretraining strategy that jointly optimize representations across both transformer layers and multiple embedding dimensions.<n>It significantly outperforms state-of-the-art embedding models in Bing-Click, a large-scale industrial retrieval dataset.
  arXiv  Detail & Related papers  (2026-05-19T09:13:49Z)
• Action-Aware Generative Sequence Modeling for Short Video Recommendation [50.74467504063892]
  Short videos often contain diverse segments, and users may not hold the same attitude toward all of them.<n>Traditional binary-classification recommendation models, which treat a video as a single holistic entity, face limitations in accurately capturing such nuanced preferences.<n>We propose a novel modeling paradigm: Action-Aware Generative Sequence Network (A2Gen), which refines user actions along the temporal dimension.
  arXiv  Detail & Related papers  (2026-04-28T16:41:04Z)
• CURP: Codebook-based Continuous User Representation for Personalized Generation with LLMs [60.867541073274715]
  We propose a novel framework CURP, which employs a bidirectional user encoder and a discrete prototype codebook to extract multi-dimensional user traits.<n>This design enables plug-and-play personalization with a small number of trainable parameters.<n>We show that CURP achieves superior performance and generalization compared to strong baselines.
  arXiv  Detail & Related papers  (2026-01-31T14:13:06Z)
• Youtu-LLM: Unlocking the Native Agentic Potential for Lightweight Large Language Models [78.73992315826035]
  We introduce Youtu-LLM, a lightweight language model that harmonizes high computational efficiency with native agentic intelligence.<n>Youtu-LLM is pre-trained from scratch to systematically cultivate reasoning and planning capabilities.
  arXiv  Detail & Related papers  (2025-12-31T04:25:11Z)
• EoS-FM: Can an Ensemble of Specialist Models act as a Generalist Feature Extractor? [8.178030486012437]
  We present an Ensemble-of-Specialists framework for building Remote Sensing Foundation Models (RSFMs)<n>Our method decomposes the training process into lightweight, task-specific ConvNeXtV2 specialists that can be frozen and reused.<n>Our framework sets a new direction for building scalable and efficient RSFMs.
  arXiv  Detail & Related papers  (2025-11-26T15:52:56Z)
• NExT-OMNI: Towards Any-to-Any Omnimodal Foundation Models with Discrete Flow Matching [64.10695425442164]
  We introduce NExT-OMNI, an open-source omnimodal foundation model that achieves unified modeling through discrete flow paradigms.<n>Trained on large-scale interleaved text, image, video, and audio data, NExT-OMNI delivers competitive performance on multimodal generation and understanding benchmarks.<n>To advance further research, we release training details, data protocols, and open-source both the code and model checkpoints.
  arXiv  Detail & Related papers  (2025-10-15T16:25:18Z)
• Realizing Scaling Laws in Recommender Systems: A Foundation-Expert Paradigm for Hyperscale Model Deployment [16.883389041355073]
  We propose a framework designed for the development and deployment of hyperscale recommendation FMs.<n>In our approach, a central FM is trained on lifelong, cross-surface, multi-modal user data to learn generalizable knowledge.<n>This knowledge is then efficiently transferred to various lightweight, surface-specific "expert" models via target-aware embeddings.
  arXiv  Detail & Related papers  (2025-08-04T22:03:13Z)
• Noise-powered Multi-modal Knowledge Graph Representation Framework [52.95468915728721]
  The rise of Multi-modal Pre-training highlights the necessity for a unified Multi-Modal Knowledge Graph representation learning framework.<n>We propose a novel SNAG method that utilizes a Transformer-based architecture equipped with modality-level noise masking.<n>Our approach achieves SOTA performance across a total of ten datasets, demonstrating its versatility.
  arXiv  Detail & Related papers  (2024-03-11T15:48:43Z)
• LiMAML: Personalization of Deep Recommender Models via Meta Learning [13.69036196446634]
  We introduce an innovative meta-learning solution tailored to the personalization of models for individual members and other entities. We leverage the Model-Agnostic Meta Learning (MAML) algorithm to adapt per-task sub-networks using recent user interaction data. Our approach has enabled the deployment of a range of highly personalized AI models across diverse LinkedIn applications.
  arXiv  Detail & Related papers  (2024-02-23T22:06:36Z)
• Scaling User Modeling: Large-scale Online User Representations for Ads Personalization in Meta [25.100342052990793]
  Scaling User Modeling (SUM) is a framework widely deployed in Meta's ads ranking system. To date, SUM has been launched to hundreds of ads ranking models in Meta, processing hundreds of billions of user requests daily.
  arXiv  Detail & Related papers  (2023-11-16T03:47:48Z)
• Empowering General-purpose User Representation with Full-life Cycle Behavior Modeling [11.698166058448555]
  We propose a novel framework called full- Life cycle User Representation Model (LURM) to tackle this challenge. LURM consists of two cascaded sub-models: (I) Bag-of-Interests (BoI) encodes user behaviors in any time period into a sparse vector with super-high dimension (e.g., 105) SMEN achieves almost dimensionality reduction, benefiting from a novel multi-anchor module which can learn different aspects of user interests.
  arXiv  Detail & Related papers  (2021-10-20T08:24:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.