Survey of User Interface Design and Interaction Techniques in Generative AI Applications

• URL: http://arxiv.org/abs/2410.22370v1
• Date: Mon, 28 Oct 2024 23:10:06 GMT
• Title: Survey of User Interface Design and Interaction Techniques in Generative AI Applications
• Authors: Reuben Luera, Ryan A. Rossi, Alexa Siu, Franck Dernoncourt, Tong Yu, Sungchul Kim, Ruiyi Zhang, Xiang Chen, Hanieh Salehy, Jian Zhao, Samyadeep Basu, Puneet Mathur, Nedim Lipka,
• Abstract summary: We aim to create a compendium of different user-interaction patterns that can be used as a reference for designers and developers alike. We also strive to lower the entry barrier for those attempting to learn more about the design of generative AI applications.
• Score: 79.55963742878684
• License:
• Abstract: The applications of generative AI have become extremely impressive, and the interplay between users and AI is even more so. Current human-AI interaction literature has taken a broad look at how humans interact with generative AI, but it lacks specificity regarding the user interface designs and patterns used to create these applications. Therefore, we present a survey that comprehensively presents taxonomies of how a human interacts with AI and the user interaction patterns designed to meet the needs of a variety of relevant use cases. We focus primarily on user-guided interactions, surveying interactions that are initiated by the user and do not include any implicit signals given by the user. With this survey, we aim to create a compendium of different user-interaction patterns that can be used as a reference for designers and developers alike. In doing so, we also strive to lower the entry barrier for those attempting to learn more about the design of generative AI applications.

Related papers

• Constraining Participation: Affordances of Feedback Features in Interfaces to Large Language Models [49.74265453289855]
  Large language models (LLMs) are now accessible to anyone with a computer, a web browser, and an internet connection via browser-based interfaces. This paper examines the affordances of interactive feedback features in ChatGPT's interface, analysing how they shape user input and participation in iteration.
  arXiv  Detail & Related papers  (2024-08-27T13:50:37Z)
• Characterizing and modeling harms from interactions with design patterns in AI interfaces [0.19116784879310028]
  We argue that design features of interfaces with adaptive AI systems can have cascading impacts, driven by feedback loops. We propose Design-Enhanced Control of AI systems (DECAI) to structure and facilitate impact assessments of AI interface designs.
  arXiv  Detail & Related papers  (2024-04-17T13:30:45Z)
• Interrogating AI: Characterizing Emergent Playful Interactions with ChatGPT [10.907980864371213]
  This study focuses on playful interactions exhibited by users of a popular AI technology, ChatGPT. We found that more than half (54%) of user discourse revolved around playful interactions. It examines how these interactions can help users understand AI's agency, shape human-AI relationships, and provide insights for designing AI systems.
  arXiv  Detail & Related papers  (2024-01-16T14:44:13Z)
• Unpacking Human-AI interactions: From interaction primitives to a design space [6.778055454461106]
  We show how these primitives can be combined into a set of interaction patterns. The motivation behind this is to provide a compact generalisation of existing practices. We discuss how this approach can be used towards a design space for Human-AI interactions.
  arXiv  Detail & Related papers  (2024-01-10T12:27:18Z)
• Enhancing HOI Detection with Contextual Cues from Large Vision-Language Models [56.257840490146]
  ConCue is a novel approach for improving visual feature extraction in HOI detection. We develop a transformer-based feature extraction module with a multi-tower architecture that integrates contextual cues into both instance and interaction detectors.
  arXiv  Detail & Related papers  (2023-11-26T09:11:32Z)
• Interactive Text Generation [75.23894005664533]
  We introduce a new Interactive Text Generation task that allows training generation models interactively without the costs of involving real users. We train our interactive models using Imitation Learning, and our experiments against competitive non-interactive generation models show that models trained interactively are superior to their non-interactive counterparts.
  arXiv  Detail & Related papers  (2023-03-02T01:57:17Z)
• Interaction Replica: Tracking Human-Object Interaction and Scene Changes From Human Motion [48.982957332374866]
  Modeling changes caused by humans is essential for building digital twins. Our method combines visual localization of humans in the scene with contact-based reasoning about human-scene interactions from IMU data. Our code, data and model are available on our project page at http://virtualhumans.mpi-inf.mpg.de/ireplica/.
  arXiv  Detail & Related papers  (2022-05-05T17:58:06Z)
• A User-Centred Framework for Explainable Artificial Intelligence in Human-Robot Interaction [70.11080854486953]
  We propose a user-centred framework for XAI that focuses on its social-interactive aspect. The framework aims to provide a structure for interactive XAI solutions thought for non-expert users.
  arXiv  Detail & Related papers  (2021-09-27T09:56:23Z)
• eXtended Artificial Intelligence: New Prospects of Human-AI Interaction Research [8.315174426992087]
  The article provides a theoretical treatment and model of human-AI interaction based on an XR-AI continuum. It shows why the combination of XR and AI fruitfully contributes to a valid and systematic investigation of human-AI interactions and interfaces. The first experiment reveals an interesting gender effect in human-robot interaction, while the second experiment reveals an Eliza effect of a recommender system.
  arXiv  Detail & Related papers  (2021-03-27T22:12:06Z)
• Player-AI Interaction: What Neural Network Games Reveal About AI as Play [14.63311356668699]
  This paper argues that games are an ideal domain for studying and experimenting with how humans interact with AI. Through a systematic survey of neural network games, we identified the dominant interaction metaphors and AI interaction patterns. Our work suggests that game and UX designers should consider flow to structure the learning curve of human-AI interaction.
  arXiv  Detail & Related papers  (2021-01-15T17:07:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.