AVIN-Chat: An Audio-Visual Interactive Chatbot System with Emotional State Tuning

• URL: http://arxiv.org/abs/2409.00012v1
• Date: Thu, 15 Aug 2024 22:45:53 GMT
• Title: AVIN-Chat: An Audio-Visual Interactive Chatbot System with Emotional State Tuning
• Authors: Chanhyuk Park, Jungbin Cho, Junwan Kim, Seongmin Lee, Jungsu Kim, Sanghoon Lee,
• Abstract summary: AVIN-Chat allows users to have face-to-face conversations with 3D avatars in real-time. The proposed AVIN-Chat emotionally speaks and expresses according to the user's emotional state.
• Score: 9.989693906734535
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This work presents an audio-visual interactive chatbot (AVIN-Chat) system that allows users to have face-to-face conversations with 3D avatars in real-time. Compared to the previous chatbot services, which provide text-only or speech-only communications, the proposed AVIN-Chat can offer audio-visual communications providing users with a superior experience quality. In addition, the proposed AVIN-Chat emotionally speaks and expresses according to the user's emotional state. Thus, it enables users to establish a strong bond with the chatbot system, increasing the user's immersion. Through user subjective tests, it is demonstrated that the proposed system provides users with a higher sense of immersion than previous chatbot systems. The demonstration video is available at https://www.youtube.com/watch?v=Z74uIV9k7_k.

Related papers

• Let's Go Real Talk: Spoken Dialogue Model for Face-to-Face Conversation [55.043492250775294]
  We introduce a novel Face-to-Face spoken dialogue model. It processes audio-visual speech from user input and generates audio-visual speech as the response. We also introduce MultiDialog, the first large-scale multimodal spoken dialogue corpus.
  arXiv  Detail & Related papers  (2024-06-12T04:48:36Z)
• InternGPT: Solving Vision-Centric Tasks by Interacting with ChatGPT Beyond Language [82.92236977726655]
  InternGPT stands for textbfinteraction, textbfnonverbal, and textbfchatbots. We present an interactive visual framework named InternGPT, or iGPT for short.
  arXiv  Detail & Related papers  (2023-05-09T17:58:34Z)
• FaceChat: An Emotion-Aware Face-to-face Dialogue Framework [58.67608580694849]
  FaceChat is a web-based dialogue framework that enables emotionally-sensitive and face-to-face conversations. System has a wide range of potential applications, including counseling, emotional support, and personalized customer service.
  arXiv  Detail & Related papers  (2023-03-08T20:45:37Z)
• ViDA-MAN: Visual Dialog with Digital Humans [50.218369825060876]
  Given a speech request, ViDA-MAN is able to response with high quality videos in sub-second latency. Backed with large knowledge base, ViDA-MAN is able to chat with users on a number of topics including chit-chat, weather, device control, News recommendations, booking hotels, as well as answering questions via structured knowledge.
  arXiv  Detail & Related papers  (2021-10-26T03:23:51Z)
• CheerBots: Chatbots toward Empathy and Emotionusing Reinforcement Learning [60.348822346249854]
  This study presents a framework whereby several empathetic chatbots are based on understanding users' implied feelings and replying empathetically for multiple dialogue turns. We call these chatbots CheerBots. CheerBots can be retrieval-based or generative-based and were finetuned by deep reinforcement learning. To respond in an empathetic way, we develop a simulating agent, a Conceptual Human Model, as aids for CheerBots in training with considerations on changes in user's emotional states in the future to arouse sympathy.
  arXiv  Detail & Related papers  (2021-10-08T07:44:47Z)
• Put Chatbot into Its Interlocutor's Shoes: New Framework to Learn Chatbot Responding with Intention [55.77218465471519]
  This paper proposes an innovative framework to train chatbots to possess human-like intentions. Our framework included a guiding robot and an interlocutor model that plays the role of humans. We examined our framework using three experimental setups and evaluate the guiding robot with four different metrics to demonstrated flexibility and performance advantages.
  arXiv  Detail & Related papers  (2021-03-30T15:24:37Z)
• Automatic Generation of Chatbots for Conversational Web Browsing [4.994942792036863]
  We describe the foundations for generating a bot out of a website equipped with simple, bot-specific annotations. The goal is to enable users to use content and functionality accessible through rendered UIs by "talking to websites" instead of by operating the graphical UI using keyboard and mouse.
  arXiv  Detail & Related papers  (2020-08-19T18:10:28Z)
• FitChat: Conversational Artificial Intelligence Interventions for Encouraging Physical Activity in Older Adults [1.8166478385879317]
  We co-created "FitChat" with older adults and we evaluate the first prototype using Think Aloud Sessions. Our thematic evaluation suggests that older adults prefer voice-based chat over text notifications or free text entry.
  arXiv  Detail & Related papers  (2020-04-29T10:39:33Z)
• "Love is as Complex as Math": Metaphor Generation System for Social Chatbot [13.128146708018438]
  We investigate the usage of a commonly used rhetorical device by human -- metaphor for social chatbots. Our work first designs a metaphor generation framework, which generates topic-aware and novel figurative sentences. Human annotators validate the novelty and properness of the generated metaphors.
  arXiv  Detail & Related papers  (2020-01-03T05:56:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.