SocialPersona: Benchmarking Personalized Profiling and Response with Multimodal Social-Media Context
- URL: http://arxiv.org/abs/2606.26654v1
- Date: Thu, 25 Jun 2026 06:31:56 GMT
- Title: SocialPersona: Benchmarking Personalized Profiling and Response with Multimodal Social-Media Context
- Abstract summary: SocialPersona is a benchmark for evaluating whether multimodal large language models (MLLMs) can recover revealed preferences from longitudinal social-media timelines and use them in dialogue.<n>Built from longitudinal timelines of 171 everyday, non-promotional social-media users, SocialPersona contains text, images, timestamps, and 2,597 human-verified preference tags across seven interest domains.
- Score: 51.206396477505116
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Personalized language-model assistants are often evaluated through a memory lens: can a model recall preferences users have explicitly stated in dialogue? More comprehensive personalization demands a harder capability -- inferring what users care about from the multimodal traces they naturally leave behind. We introduce SocialPersona, a benchmark for evaluating whether multimodal large language models (MLLMs) can recover revealed preferences from longitudinal social-media timelines and use them in dialogue. Built from longitudinal timelines of 171 everyday, non-promotional social-media users, SocialPersona contains text, images, timestamps, and 2,597 human-verified preference tags across seven interest domains, separating stable interests from recent interests. It supports two tasks: constructing structured user profiles from multimodal context and generating responses aligned with inferred profiles. Experiments with proprietary and open-weight MLLMs show that models can identify broad interest domains, yet their performance drops on fine-grained and recent interests and degrades further when inferred profiles must be used to personalize dialogue. Together with evidence that text and images provide complementary preference signals, these results indicate that robust cross-modal, long-horizon user modeling remains a key challenge, and that SocialPersona can help measure and advance progress toward assistants that infer and act on revealed preferences.
Related papers
- PERMA: Benchmarking Personalized Memory Agents via Event-Driven Preference and Realistic Task Environments [72.02445514666428]
We introduce PERMA, a benchmark designed to evaluate persona consistency over time beyond static preference recall.<n>PerMA consists of temporally ordered interaction events spanning multiple sessions and domains, with preference-related queries inserted over time.<n> Experiments demonstrate that by linking related interactions, advanced memory systems can extract more precise preferences and reduce token consumption.
arXiv Detail & Related papers (2026-03-24T14:04:11Z) - Mem-PAL: Towards Memory-based Personalized Dialogue Assistants for Long-term User-Agent Interaction [55.24448139349266]
We present PAL-Bench, a new benchmark designed to evaluate the personalization capabilities of service-oriented assistants in long-term user-agent interactions.<n>To improve personalized service-oriented interactions, we propose H$2$Memory, a hierarchical and heterogeneous memory framework.
arXiv Detail & Related papers (2025-11-17T14:22:32Z) - Post Persona Alignment for Multi-Session Dialogue Generation [16.574386220377757]
Post Persona Alignment (PPA) is a novel two-stage framework for personalized dialogue generation.<n>It first generates a general response based solely on dialogue context, then retrieves relevant persona memories using the response as a query, and finally refines the response to align with the speaker's persona.<n>Experiments on multi-session LLM-generated dialogue data demonstrate that PPA significantly outperforms prior approaches in consistency, diversity, and persona relevance.
arXiv Detail & Related papers (2025-06-13T15:04:01Z) - A Personalized Conversational Benchmark: Towards Simulating Personalized Conversations [112.81207927088117]
PersonaConvBench is a benchmark for evaluating personalized reasoning and generation in multi-turn conversations with large language models (LLMs)<n>We benchmark several commercial and open-source LLMs under a unified prompting setup and observe that incorporating personalized history yields substantial performance improvements.
arXiv Detail & Related papers (2025-05-20T09:13:22Z) - A Survey of Personalized Large Language Models: Progress and Future Directions [86.45576419251302]
Large Language Models (LLMs) excel in handling general knowledge tasks, yet struggle with user-specific personalization.<n> Personalized Large Language Models (PLLMs) tackle these challenges by leveraging individual user data.<n>PLLMs can significantly enhance user satisfaction and have broad applications in conversational agents, systems, emotion recognition, medical assistants, and more.
arXiv Detail & Related papers (2025-02-17T07:58:31Z) - Hello Again! LLM-powered Personalized Agent for Long-term Dialogue [63.65128176360345]
We introduce a model-agnostic framework, the Long-term Dialogue Agent (LD-Agent)<n>It incorporates three independently tunable modules dedicated to event perception, persona extraction, and response generation.<n>The effectiveness, generality, and cross-domain capabilities of LD-Agent are empirically demonstrated.
arXiv Detail & Related papers (2024-06-09T21:58:32Z) - PersonalityChat: Conversation Distillation for Personalized Dialog
Modeling with Facts and Traits [5.447308344436046]
PersonalityChat is a synthetic conversational dataset based upon the popular PersonaChat dataset.
We show that the personality trait labels can be used for trait-based personalization of generative dialogue models.
arXiv Detail & Related papers (2024-01-14T20:35:33Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.