Modeling Complex Behaviors: Multi-Personality Composition and Dynamic Switching in Vision-Language Models

• URL: http://arxiv.org/abs/2606.11074v2
• Date: Wed, 10 Jun 2026 03:48:53 GMT
• Title: Modeling Complex Behaviors: Multi-Personality Composition and Dynamic Switching in Vision-Language Models
• Abstract summary: This paper introduces explicit personality conditioning in Multimodal Large Language Models (MLLMs)<n>It establishes a systematic evaluation framework encompassing single-personality induction, multi-personality induction, and personality switching.<n>Experiments show that personality induction improves image captioning performance but can impair performance on tasks requiring precise reasoning.
• Score: 24.456367163695962
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the widespread deployment of Multimodal Large Language Models (MLLMs) in social interaction, understanding and controlling their behavior under complex personality conditions is essential. This paper introduces explicit personality conditioning and establishes a systematic evaluation framework encompassing single-personality induction, multi-personality induction, and personality switching. Experiments show that personality induction improves image captioning performance but can impair performance on tasks requiring precise reasoning, such as visual question answering (VQA). Balancing and residual effects are observed during multi-trait composition and dynamic switching, indicating that model behavior is co-modulated by both previous and current personality constraints. Existing prompt-based personality induction methods show limited transferability to multimodal settings. Our work reveals the dynamic and complex nature of personality modeling in MLLMs and underscores the need for robust, tailored methods for personality induction and evaluation. The code will be released when the paper is accepted.

Related papers

• Traits Run Deeper: Trait-Specific Asymmetric Fusion for Personality Assessment [53.50824432699408]
  Traits Run Deeper is a novel personality assessment framework.<n>It consists of three components: the Multimodal Foundation Representation (MFR), the Trait-Specific Modality Fusion (TSMF) and the Distribution-Calibrated Personality Regression (DCPR)
  arXiv  Detail & Related papers  (2026-06-09T06:38:36Z)
• Towards Customized Multimodal Role-Play [50.72454102691162]
  multimodal understanding and generation models enable richer human-AI interaction.<n>Yet jointly customizing a character's persona, dialogue style, and visual identity while maintaining output consistency across modalities remains largely unexplored.<n>We construct the RoleScape-20 dataset comprising 20 characters, including training and evaluation data that cover persona, stylistic descriptions, visual/expressive cues, and text-image interactions.<n>We devise UniCharacter, a two-stage training framework containing Unified Supervised Finetuning (Unified-SFT) and character-specific group relative policy optimization (Character-GRPO). Given only 10 images plus corresponding interaction examples, the model acquires the
  arXiv  Detail & Related papers  (2026-05-01T03:22:04Z)
• LLMs Reading the Rhythms of Daily Life: Aligned Understanding for Behavior Prediction and Generation [53.62804271492357]
  Large language models (LLMs) offer a promising direction due to their semantic richness, strong interpretability, and generative capabilities.<n>We propose Behavior Understanding Alignment (BUA), a novel framework that integrates LLMs into human behavior modeling through a structured curriculum learning process.<n>BUA employs sequence embeddings from pretrained behavior models as alignment anchors and guides the LLM through a three-stage curriculum, while a multi-round dialogue setting introduces prediction and generation capabilities.
  arXiv  Detail & Related papers  (2026-04-26T07:34:37Z)
• Beyond Static Personas: Situational Personality Steering for Large Language Models [52.33369223308941]
  We propose IRIS, a training-free, neuron-based Identify-Retrieve-Steer framework for advanced situational personality steering.<n>Our approach comprises situational persona neuron identification, situation-aware neuron retrieval, and similarity-weighted steering.<n>We empirically validate our framework on PersonalityBench and our newly introduced SPBench, a comprehensive situational personality benchmark.
  arXiv  Detail & Related papers  (2026-04-15T13:17:36Z)
• Experiences Build Characters: The Linguistic Origins and Functional Impact of LLM Personality [10.56924140369377]
  This study employs continued pre-training to expose models to domain-specific texts in an unsupervised manner.<n>We quantify the personality traits of model variants and analyse their relationship to linguistic style and reasoning behaviour.<n>The findings reveal that model competence is bimodal, peaking at "Expressive Generalists" and "Suppressed Specialists"
  arXiv  Detail & Related papers  (2026-03-06T09:42:41Z)
• Dynamic Personality Adaptation in Large Language Models via State Machines [1.6986898305640261]
  We propose a model-agnostic framework for dynamic personality simulation that employs state machines to represent latent personality states.<n>Part of our architecture is a modular pipeline for continuous personality scoring that evaluates dialogues along latent axes.<n>Results demonstrate that the system successfully adapts its personality state to user inputs, but also influences user behavior.
  arXiv  Detail & Related papers  (2026-02-25T18:05:11Z)
• Structured Personality Control and Adaptation for LLM Agents [11.050618253938126]
  Large Language Models (LLMs) are increasingly shaping human-computer interaction (HCI)<n>We present a framework that models LLM personality via Jungian psychological types.<n>This design allows the agent to maintain nuanced traits while dynamically adjusting to interaction demands.
  arXiv  Detail & Related papers  (2026-01-15T03:15:24Z)
• The Personality Illusion: Revealing Dissociation Between Self-Reports & Behavior in LLMs [60.15472325639723]
  Personality traits have long been studied as predictors of human behavior.<n>Recent advances in Large Language Models (LLMs) suggest similar patterns may emerge in artificial systems.
  arXiv  Detail & Related papers  (2025-09-03T21:27:10Z)
• Exploring a Gamified Personality Assessment Method through Interaction with LLM Agents Embodying Different Personalities [45.56431615835303]
  This study explores an interactive approach for personality assessment, focusing on the multiplicity of personality representation.<n>We propose a framework of Gamified Personality Assessment through Multi-Personality Representations (Multi-PR GPA)
  arXiv  Detail & Related papers  (2025-07-05T11:17:20Z)
• Personality Editing for Language Models through Adjusting Self-Referential Queries [17.051166122108857]
  We present PALETTE (Personality Adjustment by LLM SElf-TargeTed quEries), a novel method for personality editing in Large Language Models (LLMs)<n>Our approach introduces adjustment queries, where self-referential statements grounded in psychological constructs are treated analogously to factual knowledge, enabling direct editing of personality-related responses.<n>Unlike fine-tuning, PALETTE requires only 12 editing samples to achieve substantial improvements in personality alignment across personality dimensions.
  arXiv  Detail & Related papers  (2025-02-17T13:28:14Z)
• PersLLM: A Personified Training Approach for Large Language Models [66.16513246245401]
  We propose PersLLM, a framework for better data construction and model tuning.<n>For insufficient data usage, we incorporate strategies such as Chain-of-Thought prompting and anti-induction.<n>For rigid behavior patterns, we design the tuning process and introduce automated DPO to enhance the specificity and dynamism of the models' personalities.
  arXiv  Detail & Related papers  (2024-07-17T08:13:22Z)
• Editing Personality for Large Language Models [73.59001811199823]
  This paper introduces an innovative task focused on editing the personality traits of Large Language Models (LLMs) We construct PersonalityEdit, a new benchmark dataset to address this task.
  arXiv  Detail & Related papers  (2023-10-03T16:02:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.