Intent Tagging: Exploring Micro-Prompting Interactions for Supporting Granular Human-GenAI Co-Creation Workflows

• URL: http://arxiv.org/abs/2502.18737v1
• Date: Wed, 26 Feb 2025 01:13:47 GMT
• Title: Intent Tagging: Exploring Micro-Prompting Interactions for Supporting Granular Human-GenAI Co-Creation Workflows
• Authors: Frederic Gmeiner, Nicolai Marquardt, Michael Bentley, Hugo Romat, Michel Pahud, David Brown, Asta Roseway, Nikolas Martelaro, Kenneth Holstein, Ken Hinckley, Nathalie Riche,
• Abstract summary: Core challenges include misalignment of AI-generated content with user intentions (intent elicitation and alignment), user uncertainty around how to best communicate their intents to an AI system, and insufficient flexibility of AI systems to support diverse creative (workflow flexibility)<n>Motivated by these challenges, we created IntentTagger: a system for slide creation based on the notion of Intent Tags.<n>Our user study with 12 participants provides insights into the value of flexibly expressing intent across varying levels of ambiguity, meta-intent elicitation, and the benefits and challenges of intent tag-driven intent.
• Score: 28.404085240725717
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite Generative AI (GenAI) systems' potential for enhancing content creation, users often struggle to effectively integrate GenAI into their creative workflows. Core challenges include misalignment of AI-generated content with user intentions (intent elicitation and alignment), user uncertainty around how to best communicate their intents to the AI system (prompt formulation), and insufficient flexibility of AI systems to support diverse creative workflows (workflow flexibility). Motivated by these challenges, we created IntentTagger: a system for slide creation based on the notion of Intent Tags - small, atomic conceptual units that encapsulate user intent - for exploring granular and non-linear micro-prompting interactions for Human-GenAI co-creation workflows. Our user study with 12 participants provides insights into the value of flexibly expressing intent across varying levels of ambiguity, meta-intent elicitation, and the benefits and challenges of intent tag-driven workflows. We conclude by discussing the broader implications of our findings and design considerations for GenAI-supported content creation workflows.

Related papers

• Towards Machine-Generated Code for the Resolution of User Intentions [2.762180345826837]
  We investigate the feasibility of generating and collaborating through code generation that results from prompting an LLM with a concrete user intention. We provide in-depth analysis and comparison of various user intentions, the resulting code, and its execution. The employed LLM, GPT-4o-mini, exhibits remarkable proficiency in the generation of code-oriented in accordance with provided user intentions.
  arXiv  Detail & Related papers  (2025-04-24T13:19:17Z)
• Generative AI in Knowledge Work: Design Implications for Data Navigation and Decision-Making [6.460380734209551]
  We developed Yodeai, an AI-enabled system, to explore both the opportunities and limitations of AI in knowledge work. We identified three key requirements for Generative AI in knowledge work: adaptable user control, transparent collaboration mechanisms, and the ability to integrate background knowledge with external information.
  arXiv  Detail & Related papers  (2025-03-24T08:02:44Z)
• AI Automatons: AI Systems Intended to Imitate Humans [54.19152688545896]
  There is a growing proliferation of AI systems designed to mimic people's behavior, work, abilities, likenesses, or humanness. The research, design, deployment, and availability of such AI systems have prompted growing concerns about a wide range of possible legal, ethical, and other social impacts.
  arXiv  Detail & Related papers  (2025-03-04T03:55:38Z)
• Fine-Grained Appropriate Reliance: Human-AI Collaboration with a Multi-Step Transparent Decision Workflow for Complex Task Decomposition [14.413413322901409]
  We propose to investigate the impact of a novel Multi-Step Transparent (MST) decision workflow on user reliance behaviors.<n>Our findings demonstrate that human-AI collaboration with an MST decision workflow can outperform one-step collaboration in specific contexts.<n>Our work highlights that there is no one-size-fits-all decision workflow that can help obtain optimal human-AI collaboration.
  arXiv  Detail & Related papers  (2025-01-19T01:03:09Z)
• Collaborative Instance Navigation: Leveraging Agent Self-Dialogue to Minimize User Input [54.81155589931697]
  We propose a new task, Collaborative Instance Navigation (CoIN), with dynamic agent-human interaction during navigation.<n>To address CoIN, we propose a novel method, Agent-user Interaction with UncerTainty Awareness (AIUTA)<n>AIUTA achieves competitive performance in instance navigation against state-of-the-art methods, demonstrating great flexibility in handling user inputs.
  arXiv  Detail & Related papers  (2024-12-02T08:16:38Z)
• Survey of User Interface Design and Interaction Techniques in Generative AI Applications [79.55963742878684]
  We aim to create a compendium of different user-interaction patterns that can be used as a reference for designers and developers alike. We also strive to lower the entry barrier for those attempting to learn more about the design of generative AI applications.
  arXiv  Detail & Related papers  (2024-10-28T23:10:06Z)
• Data Analysis in the Era of Generative AI [56.44807642944589]
  This paper explores the potential of AI-powered tools to reshape data analysis, focusing on design considerations and challenges. We explore how the emergence of large language and multimodal models offers new opportunities to enhance various stages of data analysis workflow. We then examine human-centered design principles that facilitate intuitive interactions, build user trust, and streamline the AI-assisted analysis workflow across multiple apps.
  arXiv  Detail & Related papers  (2024-09-27T06:31:03Z)
• Content-Centric Prototyping of Generative AI Applications: Emerging Approaches and Challenges in Collaborative Software Teams [2.369736515233951]
  Our work aims to understand how collaborative software teams set up and apply design guidelines and values, iteratively prototype prompts, and evaluate prompts to achieve desired outcomes. Our findings reveal a content-centric prototyping approach in which teams begin with the content they want to generate, then identify specific attributes, constraints, and values, and explore methods to give users the ability to influence and interact with those attributes.
  arXiv  Detail & Related papers  (2024-02-27T17:56:10Z)
• Not Just Novelty: A Longitudinal Study on Utility and Customization of an AI Workflow [18.15979295351043]
  Generative AI brings novel and impressive abilities to help people in everyday tasks. It is uncertain how useful generative AI are after the novelty wears off. We conducted a three-week longitudinal study with 12 users to understand the familiarization and customization of generative AI tools for science communication.
  arXiv  Detail & Related papers  (2024-02-15T11:39:11Z)
• Exploration with Principles for Diverse AI Supervision [88.61687950039662]
  Training large transformers using next-token prediction has given rise to groundbreaking advancements in AI. While this generative AI approach has produced impressive results, it heavily leans on human supervision. This strong reliance on human oversight poses a significant hurdle to the advancement of AI innovation. We propose a novel paradigm termed Exploratory AI (EAI) aimed at autonomously generating high-quality training data.
  arXiv  Detail & Related papers  (2023-10-13T07:03:39Z)
• Octopus: Embodied Vision-Language Programmer from Environmental Feedback [58.04529328728999]
  Embodied vision-language models (VLMs) have achieved substantial progress in multimodal perception and reasoning. To bridge this gap, we introduce Octopus, an embodied vision-language programmer that uses executable code generation as a medium to connect planning and manipulation. Octopus is designed to 1) proficiently comprehend an agent's visual and textual task objectives, 2) formulate intricate action sequences, and 3) generate executable code.
  arXiv  Detail & Related papers  (2023-10-12T17:59:58Z)
• AutoML in The Wild: Obstacles, Workarounds, and Expectations [37.813441975457735]
  This study focuses on understanding the limitations of AutoML encountered by users in their real-world practices. Our findings reveal that users actively exercise user agency to overcome three major challenges arising from customizability, transparency, and privacy.
  arXiv  Detail & Related papers  (2023-02-21T17:06:46Z)
• Seamful XAI: Operationalizing Seamful Design in Explainable AI [59.89011292395202]
  Mistakes in AI systems are inevitable, arising from both technical limitations and sociotechnical gaps. We propose that seamful design can foster AI explainability by revealing sociotechnical and infrastructural mismatches. We explore this process with 43 AI practitioners and real end-users.
  arXiv  Detail & Related papers  (2022-11-12T21:54:05Z)
• Creative Wand: A System to Study Effects of Communications in Co-Creative Settings [9.356870107137093]
  Co-creative, mixed-initiative systems require user-centric means of influencing the algorithm. Key questions in co-creative AI include: How can users express their creative intentions? We introduce CREATIVE-WAND, a customizable framework for investigating co-creative mixed-initiative generation.
  arXiv  Detail & Related papers  (2022-08-04T20:56:40Z)
• The MineRL BASALT Competition on Learning from Human Feedback [58.17897225617566]
  The MineRL BASALT competition aims to spur forward research on this important class of techniques. We design a suite of four tasks in Minecraft for which we expect it will be hard to write down hardcoded reward functions. We provide a dataset of human demonstrations on each of the four tasks, as well as an imitation learning baseline.
  arXiv  Detail & Related papers  (2021-07-05T12:18:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.