The HCI Aspects of Public Deployment of Research Chatbots: A User Study,
Design Recommendations, and Open Challenges
- URL: http://arxiv.org/abs/2306.04765v1
- Date: Wed, 7 Jun 2023 20:24:43 GMT
- Title: The HCI Aspects of Public Deployment of Research Chatbots: A User Study,
Design Recommendations, and Open Challenges
- Authors: Morteza Behrooz, William Ngan, Joshua Lane, Giuliano Morse, Benjamin
Babcock, Kurt Shuster, Mojtaba Komeili, Moya Chen, Melanie Kambadur, Y-Lan
Boureau, Jason Weston
- Abstract summary: We report on a mixed-methods user study conducted on a recent research chat.
We find that abstract anthropomorphic representation for the agent has a significant effect on user's perception, that offering AI explainability may have an impact on feedback rates, and that two (diegetic and extradiegetic) levels of the chat experience should be intentionally designed.
- Score: 19.965388973809336
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Publicly deploying research chatbots is a nuanced topic involving necessary
risk-benefit analyses. While there have recently been frequent discussions on
whether it is responsible to deploy such models, there has been far less focus
on the interaction paradigms and design approaches that the resulting
interfaces should adopt, in order to achieve their goals more effectively. We
aim to pose, ground, and attempt to answer HCI questions involved in this
scope, by reporting on a mixed-methods user study conducted on a recent
research chatbot. We find that abstract anthropomorphic representation for the
agent has a significant effect on user's perception, that offering AI
explainability may have an impact on feedback rates, and that two (diegetic and
extradiegetic) levels of the chat experience should be intentionally designed.
We offer design recommendations and areas of further focus for the research
community.
Related papers
- The Impact of Human Aspects on the Interactions Between Software Developers and End-Users in Software Engineering: A Systematic Literature Review [10.307654003138401]
We present a systematic review of studies on human aspects affecting developer-user interactions.
We identified various human aspects affecting developer-user interactions in 46 studies.
Our findings suggest the importance of leveraging positive effects and addressing negative effects in developer-user interactions.
arXiv Detail & Related papers (2024-05-08T03:38:36Z) - OntoChat: a Framework for Conversational Ontology Engineering using Language Models [0.3141085922386211]
We introduce textbfOntoChat, a framework for conversational engineering that supports requirement elicitation, analysis, and testing.
We evaluate OntoChat by replicating the engineering of the Music Meta Ontology, and collecting preliminary metrics on the effectiveness of each component from users.
arXiv Detail & Related papers (2024-03-09T14:04:06Z) - Generating Human-Centric Visual Cues for Human-Object Interaction
Detection via Large Vision-Language Models [59.611697856666304]
Human-object interaction (HOI) detection aims at detecting human-object pairs and predicting their interactions.
We propose three prompts with VLM to generate human-centric visual cues within an image from multiple perspectives of humans.
We develop a transformer-based multimodal fusion module with multitower architecture to integrate visual cue features into the instance and interaction decoders.
arXiv Detail & Related papers (2023-11-26T09:11:32Z) - Leveraging Large Language Models for Automated Dialogue Analysis [12.116834890063146]
This paper investigates the ability of a state-of-the-art large language model (LLM), ChatGPT-3.5, to perform dialogue behavior detection for nine categories in real human-bot dialogues.
Our findings reveal that neither specialized models nor ChatGPT have yet achieved satisfactory results for this task, falling short of human performance.
arXiv Detail & Related papers (2023-09-12T18:03:55Z) - Ethical Aspects of ChatGPT in Software Engineering Research [4.0594888788503205]
ChatGPT can improve Software Engineering (SE) research practices by offering efficient, accessible information analysis and synthesis based on natural language interactions.
However, ChatGPT could bring ethical challenges, encompassing plagiarism, privacy, data security, and the risk of generating biased or potentially detrimental data.
This research aims to fill the given gap by elaborating on the key elements: motivators, demotivators, and ethical principles of using ChatGPT in SE research.
arXiv Detail & Related papers (2023-06-13T06:13:21Z) - Interactive Natural Language Processing [67.87925315773924]
Interactive Natural Language Processing (iNLP) has emerged as a novel paradigm within the field of NLP.
This paper offers a comprehensive survey of iNLP, starting by proposing a unified definition and framework of the concept.
arXiv Detail & Related papers (2023-05-22T17:18:29Z) - A Survey on Proactive Dialogue Systems: Problems, Methods, and Prospects [100.75759050696355]
We provide a comprehensive overview of the prominent problems and advanced designs for conversational agent's proactivity in different types of dialogues.
We discuss challenges that meet the real-world application needs but require a greater research focus in the future.
arXiv Detail & Related papers (2023-05-04T11:38:49Z) - A Categorical Archive of ChatGPT Failures [47.64219291655723]
ChatGPT, developed by OpenAI, has been trained using massive amounts of data and simulates human conversation.
It has garnered significant attention due to its ability to effectively answer a broad range of human inquiries.
However, a comprehensive analysis of ChatGPT's failures is lacking, which is the focus of this study.
arXiv Detail & Related papers (2023-02-06T04:21:59Z) - Co-Located Human-Human Interaction Analysis using Nonverbal Cues: A
Survey [71.43956423427397]
We aim to identify the nonverbal cues and computational methodologies resulting in effective performance.
This survey differs from its counterparts by involving the widest spectrum of social phenomena and interaction settings.
Some major observations are: the most often used nonverbal cue, computational method, interaction environment, and sensing approach are speaking activity, support vector machines, and meetings composed of 3-4 persons equipped with microphones and cameras, respectively.
arXiv Detail & Related papers (2022-07-20T13:37:57Z) - Investigating Human Response, Behaviour, and Preference in Joint-Task
Interaction [3.774610219328564]
We have designed an experiment in order to examine human behaviour and response as they interact with Explainable Planning (XAIP) agents.
We also present the results from an empirical analysis where we examined the behaviour of the two agents for simulated users.
arXiv Detail & Related papers (2020-11-27T22:16:59Z) - You Impress Me: Dialogue Generation via Mutual Persona Perception [62.89449096369027]
The research in cognitive science suggests that understanding is an essential signal for a high-quality chit-chat conversation.
Motivated by this, we propose P2 Bot, a transmitter-receiver based framework with the aim of explicitly modeling understanding.
arXiv Detail & Related papers (2020-04-11T12:51:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.