PEARL: Personalized Streaming Video Understanding Model

• URL: http://arxiv.org/abs/2603.20422v1
• Date: Fri, 20 Mar 2026 18:47:54 GMT
• Title: PEARL: Personalized Streaming Video Understanding Model
• Abstract summary: multimodal personalization methods are largely limited to static images or offline videos.<n>PEARL-Bench is the first comprehensive benchmark designed specifically to evaluate this challenging setting.<n>PEARL achieves state-of-the-art performance across 8 offline and online models.
• Score: 50.273809779498464
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Human cognition of new concepts is inherently a streaming process: we continuously recognize new objects or identities and update our memories over time. However, current multimodal personalization methods are largely limited to static images or offline videos. This disconnects continuous visual input from instant real-world feedback, limiting their ability to provide the real-time, interactive personalized responses essential for future AI assistants. To bridge this gap, we first propose and formally define the novel task of Personalized Streaming Video Understanding (PSVU). To facilitate research in this new direction, we introduce PEARL-Bench, the first comprehensive benchmark designed specifically to evaluate this challenging setting. It evaluates a model's ability to respond to personalized concepts at exact timestamps under two modes: (1) Frame-level, focusing on a specific person or object in discrete frames, and (2) a novel Video-level, focusing on personalized actions unfolding across continuous frames. PEARL-Bench comprises 132 unique videos and 2,173 fine-grained annotations with precise timestamps. Concept diversity and annotation quality are strictly ensured through a combined pipeline of automated generation and human verification. To tackle this challenging new setting, we further propose PEARL, a plug-and-play, training-free strategy that serves as a strong baseline. Extensive evaluations across 8 offline and online models demonstrate that PEARL achieves state-of-the-art performance. Notably, it brings consistent PSVU improvements when applied to 3 distinct architectures, proving to be a highly effective and robust strategy. We hope this work advances vision-language model (VLM) personalization and inspires further research into streaming personalized AI assistants. Code is available at https://github.com/Yuanhong-Zheng/PEARL.

Related papers

• CPC-VAR:Continual Personalized and Compositional Generation in Visual Autoregressive Models [69.9266117908314]
  Visual autoregressive ( VAR) models have emerged as efficient paradigm for text-to-image generation.<n>Existing VAR-based personalization methods fail to accommodate evolving user demands.<n>We present the first systematic study of continual personalized generation in VAR models.
  arXiv  Detail & Related papers  (2026-05-19T12:18:15Z)
• Advancing Narrative Long Video Generation via Training-Free Identity-Aware Memory [79.01059178883817]
  IAMFlow is a training-free identity-aware memory framework that explicitly models and tracks persistent entity identities.<n>A VLM asynchronously verifies and refines attributes from rendered frames, enabling explicit entity tracking in place of implicit similarity-based matching.<n>NarraStream-Bench is a benchmark for narrative streaming video generation that features 324 multi-prompt scripts spanning six dimensions.
  arXiv  Detail & Related papers  (2026-05-18T17:54:34Z)
• KlingAvatar 2.0 Technical Report [43.949604396366425]
  Our model effectively addresses the challenges of efficient, multimodally aligned long-form high-resolution video generation.<n>It delivers enhanced visual clarity, realistic lip-teeth rendering with accurate lip synchronization, strong identity preservation, and coherent multimodal instruction following.
  arXiv  Detail & Related papers  (2025-12-15T13:30:51Z)
• STARCaster: Spatio-Temporal AutoRegressive Video Diffusion for Identity- and View-Aware Talking Portraits [44.82339975771063]
  STARCaster is an identity-aware video diffusion model that addresses both speech-driven portrait animation and free-point-view talking portrait.<n>The model learns from longer temporal contexts than those generated at inference, mitigating the overly static animations common in existing autoregressive approaches.
  arXiv  Detail & Related papers  (2025-12-15T11:59:01Z)
• Skeletons Speak Louder than Text: A Motion-Aware Pretraining Paradigm for Video-Based Person Re-Identification [8.135364788458423]
  Multimodal pretraining has revolutionized visual understanding, but its impact on person-based person re-identification (ReID) remains underexplored.<n>Existing approaches often rely on video-text pairs, yet suffer from two fundamental limitations: (1) lack of genuine multimodal pretraining, and (2) text poorly captures fine-grained temporal motion.<n>We take a bold departure from text-based paradigms by introducing the first skeleton-driven pretraining framework for ReID.
  arXiv  Detail & Related papers  (2025-11-17T08:59:41Z)
• Privacy Beyond Pixels: Latent Anonymization for Privacy-Preserving Video Understanding [56.369026347458835]
  We introduce a novel formulation of visual privacy preservation for video foundation models that operates entirely in the latent space.<n>Current privacy preservation methods on input-pixel-level anonymization require retraining the entire utility video model.<n>A lightweight Anonym Adapter Module (AAM) removes private information from video features while retaining general task utility.
  arXiv  Detail & Related papers  (2025-11-11T18:56:27Z)
• Eyes Wide Open: Ego Proactive Video-LLM for Streaming Video [36.94345183020698]
  We focus on the innovative task where, given ego-streaming video input, an assistant proactively answers diverse, evolving questions at the opportune moment.<n>This task embodies three key properties: (1) Proactive Coherence, (2) Just-in-Time Responsiveness, and (3) Synchronized Efficiency.<n>We propose a comprehensive technical pipeline to enable models to tackle this challenging task.
  arXiv  Detail & Related papers  (2025-10-16T11:11:13Z)
• Understanding Long Videos via LLM-Powered Entity Relation Graphs [51.13422967711056]
  GraphVideoAgent is a framework that maps and monitors the evolving relationships between visual entities throughout the video sequence.<n>Our approach demonstrates remarkable effectiveness when tested against industry benchmarks.
  arXiv  Detail & Related papers  (2025-01-27T10:57:24Z)
• Multi-subject Open-set Personalization in Video Generation [110.02124633005516]
  We present Video Alchemist $-$ a video model with built-in multi-subject, open-set personalization capabilities.<n>Our model is built on a new Diffusion Transformer module that fuses each conditional reference image and its corresponding subject-level text prompt.<n>Our method significantly outperforms existing personalization methods in both quantitative and qualitative evaluations.
  arXiv  Detail & Related papers  (2025-01-10T18:59:54Z)
• Video-based Person Re-identification with Long Short-Term Representation Learning [101.62570747820541]
  Video-based person Re-Identification (V-ReID) aims to retrieve specific persons from raw videos captured by non-overlapped cameras. We propose a novel deep learning framework named Long Short-Term Representation Learning (LSTRL) for effective V-ReID.
  arXiv  Detail & Related papers  (2023-08-07T16:22:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.