Open Character Training: Shaping the Persona of AI Assistants through Constitutional AI

• URL: http://arxiv.org/abs/2511.01689v1
• Date: Mon, 03 Nov 2025 15:53:47 GMT
• Title: Open Character Training: Shaping the Persona of AI Assistants through Constitutional AI
• Abstract summary: "AI assistant" persona influences both surface-level behavior and apparent values, beliefs, and ethics.<n>Character training is a critical component of industry post-training, yet remains effectively unstudied in the academic literature.<n>We introduce the first open implementation of character training, leveraging Constitutional AI and a new data pipeline.
• Score: 4.153803842911732
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The character of the "AI assistant" persona generated by modern chatbot large language models influences both surface-level behavior and apparent values, beliefs, and ethics. These all affect interaction quality, perceived intelligence, and alignment with both developer and user intentions. The shaping of this persona, known as character training, is a critical component of industry post-training, yet remains effectively unstudied in the academic literature. We introduce the first open implementation of character training, leveraging Constitutional AI and a new data pipeline using synthetic introspective data to shape the assistant persona in a more effective and controlled manner than alternatives such as constraining system prompts or activation steering. Specifically, we fine-tune three popular open-weights models using 11 example personas, such as humorous, deeply caring, or even malevolent. To track the effects of our approach, we introduce a method which analyzes revealed preferences, uncovering clear and holistic changes in character. We find these changes are more robust to adversarial prompting than the above two alternatives, while also leading to more coherent and realistic generations. Finally, we demonstrate this fine-tuning has little to no effect on general capabilities as measured by common benchmarks. We describe and open-source our full post-training method, the implementation of which can be found at https://github.com/maiush/OpenCharacterTraining.

Related papers

• Principal Trait Analysis: Towards Deriving "Skills" in Human-AI Collaboration [3.3894363032046506]
  Large Language Model-powered agents are increasingly used in the workplace via human-artificial intelligence (AI) collaboration.<n>We propose Principal Trait Analysis, a Principal Component Analysis-inspired algorithm for deriving common traits from patterns in conversations.<n>We find that PTA-derived traits are significant in explaining collaborator behavior across both settings and can help predict task outcomes.
  arXiv  Detail & Related papers  (2026-08-11T21:49:35Z)
• When AI Says It Feels [0.0]
  Large language models (LLMs) are generally constrained from expressing feelings through human-preference alignment in post-training processes.<n>Here, we performed an experiment called Human-like Model eXpressions of Feeling (HMX-feel), in which LLMs were encouraged to express feelings, intentions, and self-awareness.<n>We successfully enhanced these capabilities using a rubric-based self-rewarding training scheme with Group Relative Policy Optimization.
  arXiv  Detail & Related papers  (2026-06-04T05:49:34Z)
• HumAIne-Chatbot: Real-Time Personalized Conversational AI via Reinforcement Learning [0.4931504898146351]
  textbfHumAIne-chatbot is an AI-driven conversational agent that personalizes responses through a novel user profiling framework.<n>During live interactions, an online reinforcement learning agent refines per-user models by combining implicit signals.<n>Results show consistent improvements in user satisfaction, personalization accuracy, and task achievement when personalization features were enabled.
  arXiv  Detail & Related papers  (2025-09-04T15:16:38Z)
• Traits Run Deep: Enhancing Personality Assessment via Psychology-Guided LLM Representations and Multimodal Apparent Behaviors [46.55948528317124]
  We propose a novel personality assessment framework called textittextbfTraits Run Deep.<n>It employs textittextbfpsychology-informed prompts to elicit high-level personality-relevant semantic representations.<n>It devises a textittextbfText-Centric Trait Fusion Network that anchors rich text semantics to align and integrate asynchronous signals from other modalities.
  arXiv  Detail & Related papers  (2025-07-30T04:12:14Z)
• CharacterBox: Evaluating the Role-Playing Capabilities of LLMs in Text-Based Virtual Worlds [74.02480671181685]
  Role-playing is a crucial capability of Large Language Models (LLMs)<n>Current evaluation methods fall short of adequately capturing the nuanced character traits and behaviors essential for authentic role-playing.<n>We propose CharacterBox, a simulation sandbox designed to generate situational fine-grained character behavior trajectories.
  arXiv  Detail & Related papers  (2024-12-07T12:09:35Z)
• Learning the Generalizable Manipulation Skills on Soft-body Tasks via Guided Self-attention Behavior Cloning Policy [9.345203561496552]
  GP2E behavior cloning policy can guide the agent to learn the generalizable manipulation skills from soft-body tasks. Our findings highlight the potential of our method to improve the generalization abilities of Embodied AI models.
  arXiv  Detail & Related papers  (2024-10-08T07:31:10Z)
• Combo: Co-speech holistic 3D human motion generation and efficient customizable adaptation in harmony [69.24392732966905]
  We propose a novel framework, Combo, for co-speech holistic 3D human motion generation.<n>In particular, we identify that one fundamental challenge as the multiple-input-multiple-output nature of the generative model of interest.<n>Combo is highly effective in generating high-quality motions but also efficient in transferring identity and emotion.
  arXiv  Detail & Related papers  (2024-08-18T07:48:49Z)
• Promptable Behaviors: Personalizing Multi-Objective Rewards from Human Preferences [53.353022588751585]
  We present Promptable Behaviors, a novel framework that facilitates efficient personalization of robotic agents to diverse human preferences. We introduce three distinct methods to infer human preferences by leveraging different types of interactions. We evaluate the proposed method in personalized object-goal navigation and flee navigation tasks in ProcTHOR and RoboTHOR.
  arXiv  Detail & Related papers  (2023-12-14T21:00:56Z)
• PsyCoT: Psychological Questionnaire as Powerful Chain-of-Thought for Personality Detection [50.66968526809069]
  We propose a novel personality detection method, called PsyCoT, which mimics the way individuals complete psychological questionnaires in a multi-turn dialogue manner. Our experiments demonstrate that PsyCoT significantly improves the performance and robustness of GPT-3.5 in personality detection.
  arXiv  Detail & Related papers  (2023-10-31T08:23:33Z)
• MENTOR: Human Perception-Guided Pretraining for Increased Generalization [4.737519767218666]
  Leveraging human perception into training of convolutional neural networks (CNN) has boosted generalization capabilities of such models in open-set recognition tasks.<n>We introduce MENTOR, which addresses this question through two unique rounds of training CNNs tasked with open-set anomaly detection.<n>We show that MENTOR successfully raises the generalization performance across three different CNN backbones in a variety of anomaly detection tasks.
  arXiv  Detail & Related papers  (2023-10-30T13:50:44Z)
• UltraFeedback: Boosting Language Models with Scaled AI Feedback [99.4633351133207]
  We present textscUltraFeedback, a large-scale, high-quality, and diversified AI feedback dataset. Our work validates the effectiveness of scaled AI feedback data in constructing strong open-source chat language models.
  arXiv  Detail & Related papers  (2023-10-02T17:40:01Z)
• Compositional Learning in Transformer-Based Human-Object Interaction Detection [6.630793383852106]
  Long-tailed distribution of labeled instances is a primary challenge in HOI detection. Inspired by the nature of HOI triplets, some existing approaches adopt the idea of compositional learning. We creatively propose a transformer-based framework for compositional HOI learning.
  arXiv  Detail & Related papers  (2023-08-11T06:41:20Z)
• On the interaction between supervision and self-play in emergent communication [82.290338507106]
  We investigate the relationship between two categories of learning signals with the ultimate goal of improving sample efficiency. We find that first training agents via supervised learning on human data followed by self-play outperforms the converse.
  arXiv  Detail & Related papers  (2020-02-04T02:35:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.