Building a User Foundation Model for the Open Web

• URL: http://arxiv.org/abs/2607.28019v1
• Date: Thu, 30 Jul 2026 11:07:54 GMT
• Title: Building a User Foundation Model for the Open Web
• Abstract summary: We present a user foundation model that applies self-supervised learning on user browsing histories.<n>We show that the learned representation improves multiple downstream production tasks.
• Score: 2.097346573871652
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: User foundation models have demonstrated strong results in e-commerce and social recommendation, but most industrial deployments assume environments where user identity is stable and persistent. Open-web real-time bidding (RTB) operates on a structurally different data distribution: user identity is fragmented and non-persistent across browsing sessions, and the availability of browsing history depends on user privacy choices. Consequently, a significant portion of traffic carries no historical data, and available records often consist of relatively short, disjointed sessions. As a result, historical signals in this domain are typically represented as aggregated counters and recency buckets, leaving the sequential structure unexploited. To address this limitation, we present a user foundation model that applies self-supervised learning on user browsing histories and show that the learned representation improves multiple downstream production tasks, demonstrating the viability of this approach on the open web. We pre-train a Transformer encoder with masked language modeling and a sequence-level contrastive objective, then fine-tune it on the click prediction task. We optimize the encoder's pre-training pipeline with an LLM-in-the-loop search over a curated catalog of reviewable, code-level edits (lifters), instantiating the LLM-as-optimizer paradigm in an industrial setting. The same encoder representation yields +1.197% RIG on the production bid win-rate model and +1.354% RIG on the production CTR ranker; a 7-day live A/B test confirms +2.13% CTR, -1.13% eCPC (80% CI excluding zero on both metrics).

Related papers

• From Clicks to Intent: Cross-Platform Session Embeddings with LLM-Distilled Taxonomy for Financial Services Recommendations [35.562512604326]
  We propose a scalable and dual-purpose intent prediction framework for web-based interactions.<n>Our approach transforms raw web clickstreams into two outputs: a self-supervised Transformer encodes multi-modal clickstreams into a compact session embedding.<n>Our system demonstrates that self-supervised clickstream representations combined with LLM-distilled can jointly serve quantitative tasks and qualitative understanding in production.
  arXiv  Detail & Related papers  (2026-06-24T18:18:56Z)
• TokenMinds: Pretrained User Tokens and Embeddings for User Understanding in Large Recommender Systems [22.086361560657593]
  TokenMinds is an industrial-scale system that extends the PLUM framework from item retrieval to user modeling.<n>It generates both discrete SID-based user tokens and dense user embeddings via an encoder-decoder architecture adapted from pre-trained LLMs.
  arXiv  Detail & Related papers  (2026-06-23T20:20:37Z)
• Dooly: Configuration-Agnostic, Redundancy-Aware Profiling for LLM Inference Simulation [7.365191900032772]
  Profile-based simulators are the standard tool, yet they hardcode their operation set to a specific configuration and re-profile every operation from scratch.<n>This cost stems from a missing structural understanding: every input dimension of each operation is fixed by the model configuration or determined by the incoming request.<n>We present Dooly, which exploits this structure to achieve configuration-agnostic, redundancy-aware profiling.
  arXiv  Detail & Related papers  (2026-05-08T16:44:47Z)
• Action-Aware Generative Sequence Modeling for Short Video Recommendation [50.74467504063892]
  Short videos often contain diverse segments, and users may not hold the same attitude toward all of them.<n>Traditional binary-classification recommendation models, which treat a video as a single holistic entity, face limitations in accurately capturing such nuanced preferences.<n>We propose a novel modeling paradigm: Action-Aware Generative Sequence Network (A2Gen), which refines user actions along the temporal dimension.
  arXiv  Detail & Related papers  (2026-04-28T16:41:04Z)
• Query as Anchor: Scenario-Adaptive User Representation via Large Language Model [28.30329175937291]
  We propose Query-as-Anchor, a framework shifting user modeling from static encoding to dynamic, query-aware synthesis.<n>We first construct UserU, an industrial-scale pre-training dataset that aligns behavioral sequences with user understanding semantics.<n>We introduce Cluster-based Soft Prompt Tuning to enforce discriminative latent structures.<n>For deployment, anchoring queries at sequence termini enables KV-cache-accelerated inference with negligible incremental latency.
  arXiv  Detail & Related papers  (2026-02-16T06:09:31Z)
• GenCI: Generative Modeling of User Interest Shift via Cohort-based Intent Learning for CTR Prediction [84.0125708499372]
  We propose a generative user intent framework to model user preferences for click-through rate (CTR) prediction.<n>The framework first employs a generative model, trained with a next-item prediction objective, to proactively produce candidate interest cohorts.<n>A hierarchical candidate-aware network then injects this rich contextual signal into the ranking stage, refining them with cross-attention to align with both user history and the target item.
  arXiv  Detail & Related papers  (2026-01-26T08:15:04Z)
• SessionIntentBench: A Multi-task Inter-session Intention-shift Modeling Benchmark for E-commerce Customer Behavior Understanding [64.45047674586671]
  We introduce the concept of an intention tree and propose a dataset curation pipeline.<n>We construct a sibling multimodal benchmark, SessionIntentBench, that evaluates L(V)LMs' capability on understanding inter-session intention shift.<n>With 1,952,177 intention entries, 1,132,145 session intention trajectories, and 13,003,664 available tasks mined using 10,905 sessions, we provide a scalable way to exploit the existing session data.
  arXiv  Detail & Related papers  (2025-07-27T09:04:17Z)
• MMGRec: Multimodal Generative Recommendation with Transformer Model [81.61896141495144]
  MMGRec aims to introduce a generative paradigm into multimodal recommendation. We first devise a hierarchical quantization method Graph CF-RQVAE to assign Rec-ID for each item from its multimodal information. We then train a Transformer-based recommender to generate the Rec-IDs of user-preferred items based on historical interaction sequences.
  arXiv  Detail & Related papers  (2024-04-25T12:11:27Z)
• Breaking the Length Barrier: LLM-Enhanced CTR Prediction in Long Textual User Behaviors [25.086118164540974]
  Large language models (LLMs) are used to improve the performance of click-through rate (CTR) prediction. As user sequences grow longer, the current efficiency of LLMs is inadequate for training on billions of users and items. We propose Behavior Aggregated Hierarchical (BAHE) to enhance the efficiency of LLM-based CTR modeling.
  arXiv  Detail & Related papers  (2024-03-28T12:05:15Z)
• FedAvg with Fine Tuning: Local Updates Lead to Representation Learning [54.65133770989836]
  Federated Averaging (FedAvg) algorithm consists of alternating between a few local gradient updates at client nodes, followed by a model averaging update at the server. We show that the reason behind generalizability of the FedAvg's output is its power in learning the common data representation among the clients' tasks. We also provide empirical evidence demonstrating FedAvg's representation learning ability in federated image classification with heterogeneous data.
  arXiv  Detail & Related papers  (2022-05-27T00:55:24Z)
• End-to-End Object Detection with Transformers [88.06357745922716]
  We present a new method that views object detection as a direct set prediction problem. Our approach streamlines the detection pipeline, effectively removing the need for many hand-designed components. The main ingredients of the new framework, called DEtection TRansformer or DETR, are a set-based global loss.
  arXiv  Detail & Related papers  (2020-05-26T17:06:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.