Learning Japanese with Jouzu: Interaction Outcomes with Stylized Dialogue Fictional Agents

• URL: http://arxiv.org/abs/2507.06483v1
• Date: Wed, 09 Jul 2025 01:57:58 GMT
• Title: Learning Japanese with Jouzu: Interaction Outcomes with Stylized Dialogue Fictional Agents
• Authors: Zackary Rackauckas, Julia Hirschberg,
• Abstract summary: This study investigates how stylized, voiced agents shape user interaction in a multimodal language learning environment.<n>We conducted a mixed-methods evaluation of 54 participants interacting with anime-inspired characters powered by large language models and expressive text-to-speech synthesis.<n>Our findings reveal that agent design, especially voice, persona, and linguistic style, substantially affected user experience, motivation, and strategy.
• Score: 4.740589102992697
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: This study investigates how stylized, voiced agents shape user interaction in a multimodal language learning environment. We conducted a mixed-methods evaluation of 54 participants interacting with anime-inspired characters powered by large language models and expressive text-to-speech synthesis. These agents responded in Japanese character language, offering users asynchronous, semi-structured conversation in varying speech styles and emotional tones. We analyzed user engagement patterns, perceived usability, emotional responses, and learning behaviors, with particular attention to how agent stylization influenced interaction across language proficiency levels and cultural backgrounds. Our findings reveal that agent design, especially voice, persona, and linguistic style, substantially affected user experience, motivation, and strategy. This work contributes to the understanding of affective, culturally stylized agents in human-agent interaction and offers guidance for designing more engaging, socially responsive systems.

Related papers

• Aligning Spoken Dialogue Models from User Interactions [55.192134724622235]
  We propose a novel preference alignment framework to improve spoken dialogue models on realtime conversations from user interactions.<n>We create a dataset of more than 150,000 preference pairs from raw multi-turn speech conversations annotated with AI feedback.<n>Our findings shed light on the importance of a well-calibrated balance among various dynamics, crucial for natural real-time speech dialogue systems.
  arXiv  Detail & Related papers  (2025-06-26T16:45:20Z)
• From Persona to Person: Enhancing the Naturalness with Multiple Discourse Relations Graph Learning in Personalized Dialogue Generation [11.442761234901289]
  We propose MUDI ($textbfMu$ltiple $textbfDi$scourse Relations Graph Learning) for personalized dialogue generation.<n>We utilize a Large Language Model to assist in annotating discourse relations and to transform dialogue data into structured dialogue graphs.<n>Our experiments demonstrate significant improvements in the quality of personalized responses, thus resembling human-like dialogue exchanges.
  arXiv  Detail & Related papers  (2025-06-13T08:12:52Z)
• OmniCharacter: Towards Immersive Role-Playing Agents with Seamless Speech-Language Personality Interaction [123.89581506075461]
  We propose OmniCharacter, a first seamless speech-language personality interaction model to achieve immersive RPAs with low latency.<n> Specifically, OmniCharacter enables agents to consistently exhibit role-specific personality traits and vocal traits throughout the interaction.<n>Our method yields better responses in terms of both content and style compared to existing RPAs and mainstream speech-language models, with a response latency as low as 289ms.
  arXiv  Detail & Related papers  (2025-05-26T17:55:06Z)
• Towards Developmentally Plausible Rewards: Communicative Success as a Learning Signal for Interactive Language Models [49.22720751953838]
  We propose a method for training language models in an interactive setting inspired by child language acquisition.<n>In our setting, a speaker attempts to communicate some information to a listener in a single-turn dialogue and receives a reward if communicative success is achieved.
  arXiv  Detail & Related papers  (2025-05-09T11:48:36Z)
• Speaker effects in spoken language comprehension [0.9514940899499753]
  The identity of a speaker significantly influences spoken language comprehension by affecting both perception and expectation.<n>We propose an integrative model featuring the interplay between bottom-up perception-based processes driven by acoustic details and top-down expectation-based processes driven by a speaker model.
  arXiv  Detail & Related papers  (2024-12-10T07:03:06Z)
• Talk With Human-like Agents: Empathetic Dialogue Through Perceptible Acoustic Reception and Reaction [23.115506530649988]
  PerceptiveAgent is an empathetic multi-modal dialogue system designed to discern deeper or more subtle meanings. PerceptiveAgent perceives acoustic information from input speech and generates empathetic responses based on speaking styles described in natural language.
  arXiv  Detail & Related papers  (2024-06-18T15:19:51Z)
• CloChat: Understanding How People Customize, Interact, and Experience Personas in Large Language Models [15.915071948354466]
  CloChat is an interface supporting easy and accurate customization of agent personas in large language models. Results indicate that participants formed emotional bonds with the customized agents, engaged in more dynamic dialogues, and showed interest in sustaining interactions.
  arXiv  Detail & Related papers  (2024-02-23T11:25:17Z)
• deep learning of segment-level feature representation for speech emotion recognition in conversations [9.432208348863336]
  We propose a conversational speech emotion recognition method to deal with capturing attentive contextual dependency and speaker-sensitive interactions. First, we use a pretrained VGGish model to extract segment-based audio representation in individual utterances. Second, an attentive bi-directional recurrent unit (GRU) models contextual-sensitive information and explores intra- and inter-speaker dependencies jointly.
  arXiv  Detail & Related papers  (2023-02-05T16:15:46Z)
• Few-shot Language Coordination by Modeling Theory of Mind [95.54446989205117]
  We study the task of few-shot $textitlanguage coordination$. We require the lead agent to coordinate with a $textitpopulation$ of agents with different linguistic abilities. This requires the ability to model the partner's beliefs, a vital component of human communication.
  arXiv  Detail & Related papers  (2021-07-12T19:26:11Z)
• Can You be More Social? Injecting Politeness and Positivity into Task-Oriented Conversational Agents [60.27066549589362]
  Social language used by human agents is associated with greater users' responsiveness and task completion. The model uses a sequence-to-sequence deep learning architecture, extended with a social language understanding element. Evaluation in terms of content preservation and social language level using both human judgment and automatic linguistic measures shows that the model can generate responses that enable agents to address users' issues in a more socially appropriate way.
  arXiv  Detail & Related papers  (2020-12-29T08:22:48Z)
• XPersona: Evaluating Multilingual Personalized Chatbot [76.00426517401894]
  We propose a multi-lingual extension of Persona-Chat, namely XPersona. Our dataset includes persona conversations in six different languages other than English for building and evaluating multilingual personalized agents.
  arXiv  Detail & Related papers  (2020-03-17T07:52:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.