AACessTalk: Fostering Communication between Minimally Verbal Autistic Children and Parents with Contextual Guidance and Card Recommendation

• URL: http://arxiv.org/abs/2409.09641v2
• Date: Tue, 17 Sep 2024 14:16:16 GMT
• Title: AACessTalk: Fostering Communication between Minimally Verbal Autistic Children and Parents with Contextual Guidance and Card Recommendation
• Authors: Dasom Choi, SoHyun Park, Kyungah Lee, Hwajung Hong, Young-Ho Kim,
• Abstract summary: We present AACessTalk, a tablet-based, AI-mediated communication system. It facilitates meaningful exchanges between an MVA child and a parent.
• Score: 17.30104178658932
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: As minimally verbal autistic (MVA) children communicate with parents through few words and nonverbal cues, parents often struggle to encourage their children to express subtle emotions and needs and to grasp their nuanced signals. We present AACessTalk, a tablet-based, AI-mediated communication system that facilitates meaningful exchanges between an MVA child and a parent. AACessTalk provides real-time guides to the parent to engage the child in conversation and, in turn, recommends contextual vocabulary cards to the child. Through a two-week deployment study with 11 MVA child-parent dyads, we examine how AACessTalk fosters everyday conversation practice and mutual engagement. Our findings show high engagement from all dyads, leading to increased frequency of conversation and turn-taking. AACessTalk also encouraged parents to explore their own interaction strategies and empowered the children to have more agency in communication. We discuss the implications of designing technologies for balanced communication dynamics in parent-MVA child interaction.

Related papers

• Understanding Parents' Desires in Moderating Children's Interactions with GenAI Chatbots through LLM-Generated Probes [23.371075498013457]
  This paper studies how parents want to moderate children's interactions with Generative AI chatbots.<n>Our findings reveal three key insights: (1) parents express concern about interactions that current GenAI parental controls neglect; (2) parents want fine-grained transparency and moderation at the conversation level; and (3) parents need personalized controls that adapt to their desired strategies and children's ages.
  arXiv  Detail & Related papers  (2026-03-04T05:00:14Z)
• TAVID: Text-Driven Audio-Visual Interactive Dialogue Generation [72.46711449668814]
  We introduce TAVID, a unified framework that generates both interactive faces and conversational speech in a synchronized manner.<n>We evaluate our system across four dimensions: talking face realism, listening head responsiveness, dyadic interaction, and speech quality.
  arXiv  Detail & Related papers  (2025-12-23T12:04:23Z)
• Teacher Demonstrations in a BabyLM's Zone of Proximal Development for Contingent Multi-Turn Interaction [8.154459044756367]
  We introduce ContingentChat, a framework that benchmarks and improves multi-turn contingency in a BabyLM trained on 100M words.<n>Experiments with adaptive teacher decoding strategies show limited additional gains.
  arXiv  Detail & Related papers  (2025-10-23T10:29:23Z)
• Autiverse: Eliciting Autistic Adolescents' Daily Narratives through AI-guided Multimodal Journaling [21.638838467146467]
  We present Autiverse, an AI-guided multimodal journaling app for tablets.<n>Autiverse elicits key details through a stepwise dialogue with peer-like, customizable AI.<n>It composes them into an editable four-panel comic strip.
  arXiv  Detail & Related papers  (2025-09-22T08:02:09Z)
• Designing for Engaging Communication Between Parents and Young Adult Children Through Shared Music Experiences [6.329321597138646]
  We develop DJ-Fam, a mobile application that enables parents and children to listen to their favorite songs and use them as conversation starters.<n>From our deployment study with seven families over four weeks in South Korea, we show the potential of DJ-Fam to influence parent-child interaction positively.
  arXiv  Detail & Related papers  (2025-07-30T16:34:44Z)
• SingaKids: A Multilingual Multimodal Dialogic Tutor for Language Learning [33.91186948786452]
  We introduce SingaKids, a dialogic tutor designed to facilitate language learning through picture description tasks.<n>Our system integrates dense image captioning, multilingual dialogic interaction, speech understanding, and engaging speech generation to create an immersive learning environment.
  arXiv  Detail & Related papers  (2025-06-03T03:56:45Z)
• CoVoL: A Cooperative Vocabulary Learning Game for Children with Autism [1.3155469104700699]
  We propose the design of a cooperative two-player vocabulary learning game, CoVoL.<n>CoVoL allows children to engage in game-based vocabulary learning useful for real-world social communication scenarios.
  arXiv  Detail & Related papers  (2025-05-13T12:48:02Z)
• Towards Developmentally Plausible Rewards: Communicative Success as a Learning Signal for Interactive Language Models [49.22720751953838]
  We propose a method for training language models in an interactive setting inspired by child language acquisition.<n>In our setting, a speaker attempts to communicate some information to a listener in a single-turn dialogue and receives a reward if communicative success is achieved.
  arXiv  Detail & Related papers  (2025-05-09T11:48:36Z)
• Interactive Dialogue Agents via Reinforcement Learning on Hindsight Regenerations [58.65755268815283]
  Many real dialogues are interactive, meaning an agent's utterances will influence their conversational partner, elicit information, or change their opinion. We use this fact to rewrite and augment existing suboptimal data, and train via offline reinforcement learning (RL) an agent that outperforms both prompting and learning from unaltered human demonstrations. Our results in a user study with real humans show that our approach greatly outperforms existing state-of-the-art dialogue agents.
  arXiv  Detail & Related papers  (2024-11-07T21:37:51Z)
• Egocentric Speaker Classification in Child-Adult Dyadic Interactions: From Sensing to Computational Modeling [30.099739460287566]
  Autism spectrum disorder (ASD) is a neurodevelopmental condition characterized by challenges in social communication, repetitive behavior, and sensory processing. One important research area in ASD is evaluating children's behavioral changes over time during treatment. A fundamental aspect of understanding children's behavior in these interactions is automatic speech understanding.
  arXiv  Detail & Related papers  (2024-09-14T07:03:08Z)
• TwIPS: A Large Language Model Powered Texting Application to Simplify Conversational Nuances for Autistic Users [0.0]
  Autistic individuals often experience difficulties in conveying and interpreting emotional tone and non-literal nuances. We present TwIPS, a prototype texting application powered by a large language model (LLM) We leverage an AI-based simulation and a conversational script to evaluate TwIPS with 8 autistic participants in an in-lab setting.
  arXiv  Detail & Related papers  (2024-07-25T04:15:54Z)
• GOMA: Proactive Embodied Cooperative Communication via Goal-Oriented Mental Alignment [72.96949760114575]
  We propose a novel cooperative communication framework, Goal-Oriented Mental Alignment (GOMA) GOMA formulates verbal communication as a planning problem that minimizes the misalignment between parts of agents' mental states that are relevant to the goals. We evaluate our approach against strong baselines in two challenging environments, Overcooked (a multiplayer game) and VirtualHome (a household simulator)
  arXiv  Detail & Related papers  (2024-03-17T03:52:52Z)
• Exploring Parent's Needs for Children-Centered AI to Support Preschoolers' Interactive Storytelling and Reading Activities [52.828843153565984]
  AI-based storytelling and reading technologies are becoming increasingly ubiquitous in preschoolers' lives. This paper investigates how they function in practical storytelling and reading scenarios and, how parents, the most critical stakeholders, experience and perceive them. Our findings suggest that even though AI-based storytelling and reading technologies provide more immersive and engaging interaction, they still cannot meet parents' expectations due to a series of interactive and algorithmic challenges.
  arXiv  Detail & Related papers  (2024-01-24T20:55:40Z)
• "Alexa doesn't have that many feelings": Children's understanding of AI through interactions with smart speakers in their homes [0.0]
  Children's understanding of AI-supported technology has educational implications. Findings will enable educators to develop appropriate materials to address the pressing need for AI literacy.
  arXiv  Detail & Related papers  (2023-05-09T16:39:34Z)
• StoryBuddy: A Human-AI Collaborative Chatbot for Parent-Child Interactive Storytelling with Flexible Parental Involvement [61.47157418485633]
  We developed StoryBuddy, an AI-enabled system for parents to create interactive storytelling experiences. A user study validated StoryBuddy's usability and suggested design insights for future parent-AI collaboration systems.
  arXiv  Detail & Related papers  (2022-02-13T04:53:28Z)
• Few-shot Language Coordination by Modeling Theory of Mind [95.54446989205117]
  We study the task of few-shot $textitlanguage coordination$. We require the lead agent to coordinate with a $textitpopulation$ of agents with different linguistic abilities. This requires the ability to model the partner's beliefs, a vital component of human communication.
  arXiv  Detail & Related papers  (2021-07-12T19:26:11Z)
• Reinforcement Learning for Emotional Text-to-Speech Synthesis with Improved Emotion Discriminability [82.39099867188547]
  Emotional text-to-speech synthesis (ETTS) has seen much progress in recent years. We propose a new interactive training paradigm for ETTS, denoted as i-ETTS. We formulate an iterative training strategy with reinforcement learning to ensure the quality of i-ETTS optimization.
  arXiv  Detail & Related papers  (2021-04-03T13:52:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.