The HCI Aspects of Public Deployment of Research Chatbots: A User Study, Design Recommendations, and Open Challenges

• URL: http://arxiv.org/abs/2306.04765v1
• Date: Wed, 7 Jun 2023 20:24:43 GMT
• Title: The HCI Aspects of Public Deployment of Research Chatbots: A User Study, Design Recommendations, and Open Challenges
• Authors: Morteza Behrooz, William Ngan, Joshua Lane, Giuliano Morse, Benjamin Babcock, Kurt Shuster, Mojtaba Komeili, Moya Chen, Melanie Kambadur, Y-Lan Boureau, Jason Weston
• Abstract summary: We report on a mixed-methods user study conducted on a recent research chat. We find that abstract anthropomorphic representation for the agent has a significant effect on user's perception, that offering AI explainability may have an impact on feedback rates, and that two (diegetic and extradiegetic) levels of the chat experience should be intentionally designed.
• Score: 19.965388973809336
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Publicly deploying research chatbots is a nuanced topic involving necessary risk-benefit analyses. While there have recently been frequent discussions on whether it is responsible to deploy such models, there has been far less focus on the interaction paradigms and design approaches that the resulting interfaces should adopt, in order to achieve their goals more effectively. We aim to pose, ground, and attempt to answer HCI questions involved in this scope, by reporting on a mixed-methods user study conducted on a recent research chatbot. We find that abstract anthropomorphic representation for the agent has a significant effect on user's perception, that offering AI explainability may have an impact on feedback rates, and that two (diegetic and extradiegetic) levels of the chat experience should be intentionally designed. We offer design recommendations and areas of further focus for the research community.

Related papers

• Survey of User Interface Design and Interaction Techniques in Generative AI Applications [79.55963742878684]
  We aim to create a compendium of different user-interaction patterns that can be used as a reference for designers and developers alike. We also strive to lower the entry barrier for those attempting to learn more about the design of generative AI applications.
  arXiv  Detail & Related papers  (2024-10-28T23:10:06Z)
• PersonaFlow: Boosting Research Ideation with LLM-Simulated Expert Personas [12.593617990325528]
  We introduce PersonaFlow, an LLM-based system using persona simulation to support research ideation. Our findings indicate that using multiple personas during ideation significantly enhances user-perceived quality of outcomes. Users' persona customization interactions significantly improved their sense of control and recall of generated ideas.
  arXiv  Detail & Related papers  (2024-09-19T07:54:29Z)
• Constraining Participation: Affordances of Feedback Features in Interfaces to Large Language Models [49.74265453289855]
  Large language models (LLMs) are now accessible to anyone with a computer, a web browser, and an internet connection via browser-based interfaces. This paper examines the affordances of interactive feedback features in ChatGPT's interface, analysing how they shape user input and participation in iteration.
  arXiv  Detail & Related papers  (2024-08-27T13:50:37Z)
• The Impact of Human Aspects on the Interactions Between Software Developers and End-Users in Software Engineering: A Systematic Literature Review [10.307654003138401]
  We present a systematic review of studies on human aspects affecting developer-user interactions. We identified various human aspects affecting developer-user interactions in 46 studies. Our findings suggest the importance of leveraging positive effects and addressing negative effects in developer-user interactions.
  arXiv  Detail & Related papers  (2024-05-08T03:38:36Z)
• Enhancing HOI Detection with Contextual Cues from Large Vision-Language Models [56.257840490146]
  ConCue is a novel approach for improving visual feature extraction in HOI detection. We develop a transformer-based feature extraction module with a multi-tower architecture that integrates contextual cues into both instance and interaction detectors.
  arXiv  Detail & Related papers  (2023-11-26T09:11:32Z)
• Leveraging Large Language Models for Automated Dialogue Analysis [12.116834890063146]
  This paper investigates the ability of a state-of-the-art large language model (LLM), ChatGPT-3.5, to perform dialogue behavior detection for nine categories in real human-bot dialogues. Our findings reveal that neither specialized models nor ChatGPT have yet achieved satisfactory results for this task, falling short of human performance.
  arXiv  Detail & Related papers  (2023-09-12T18:03:55Z)
• Ethical Aspects of ChatGPT in Software Engineering Research [4.0594888788503205]
  ChatGPT can improve Software Engineering (SE) research practices by offering efficient, accessible information analysis and synthesis based on natural language interactions. However, ChatGPT could bring ethical challenges, encompassing plagiarism, privacy, data security, and the risk of generating biased or potentially detrimental data. This research aims to fill the given gap by elaborating on the key elements: motivators, demotivators, and ethical principles of using ChatGPT in SE research.
  arXiv  Detail & Related papers  (2023-06-13T06:13:21Z)
• Interactive Natural Language Processing [67.87925315773924]
  Interactive Natural Language Processing (iNLP) has emerged as a novel paradigm within the field of NLP. This paper offers a comprehensive survey of iNLP, starting by proposing a unified definition and framework of the concept.
  arXiv  Detail & Related papers  (2023-05-22T17:18:29Z)
• A Survey on Proactive Dialogue Systems: Problems, Methods, and Prospects [100.75759050696355]
  We provide a comprehensive overview of the prominent problems and advanced designs for conversational agent's proactivity in different types of dialogues. We discuss challenges that meet the real-world application needs but require a greater research focus in the future.
  arXiv  Detail & Related papers  (2023-05-04T11:38:49Z)
• A Categorical Archive of ChatGPT Failures [47.64219291655723]
  ChatGPT, developed by OpenAI, has been trained using massive amounts of data and simulates human conversation. It has garnered significant attention due to its ability to effectively answer a broad range of human inquiries. However, a comprehensive analysis of ChatGPT's failures is lacking, which is the focus of this study.
  arXiv  Detail & Related papers  (2023-02-06T04:21:59Z)
• Investigating Human Response, Behaviour, and Preference in Joint-Task Interaction [3.774610219328564]
  We have designed an experiment in order to examine human behaviour and response as they interact with Explainable Planning (XAIP) agents. We also present the results from an empirical analysis where we examined the behaviour of the two agents for simulated users.
  arXiv  Detail & Related papers  (2020-11-27T22:16:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.