Content-Centric Prototyping of Generative AI Applications: Emerging Approaches and Challenges in Collaborative Software Teams

• URL: http://arxiv.org/abs/2402.17721v1
• Date: Tue, 27 Feb 2024 17:56:10 GMT
• Title: Content-Centric Prototyping of Generative AI Applications: Emerging Approaches and Challenges in Collaborative Software Teams
• Authors: Hari Subramonyam, Divy Thakkar, J\"urgen Dieber, Anoop Sinha
• Abstract summary: Our work aims to understand how collaborative software teams set up and apply design guidelines and values, iteratively prototype prompts, and evaluate prompts to achieve desired outcomes. Our findings reveal a content-centric prototyping approach in which teams begin with the content they want to generate, then identify specific attributes, constraints, and values, and explore methods to give users the ability to influence and interact with those attributes.
• Score: 2.369736515233951
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generative AI models are increasingly powering software applications, offering the capability to produce expressive content across varied contexts. However, unlike previous iterations of human-AI design, the emerging design process for generative capabilities primarily hinges on prompt engineering strategies. Given this fundamental shift in approach, our work aims to understand how collaborative software teams set up and apply design guidelines and values, iteratively prototype prompts, and evaluate prompts to achieve desired outcomes. We conducted design studies with 39 industry professionals, including designers, software engineers, and product managers. Our findings reveal a content-centric prototyping approach in which teams begin with the content they want to generate, then identify specific attributes, constraints, and values, and explore methods to give users the ability to influence and interact with those attributes. Based on associated challenges, such as the lack of model interpretability and overfitting the design to examples, we outline considerations for generative AI prototyping.

Related papers

• A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems [93.8285345915925]
  Reasoning is a fundamental cognitive process that enables logical inference, problem-solving, and decision-making. With the rapid advancement of large language models (LLMs), reasoning has emerged as a key capability that distinguishes advanced AI systems. We categorize existing methods along two dimensions: (1) Regimes, which define the stage at which reasoning is achieved; and (2) Architectures, which determine the components involved in the reasoning process.
  arXiv  Detail & Related papers  (2025-04-12T01:27:49Z)
• AI Agents in Engineering Design: A Multi-Agent Framework for Aesthetic and Aerodynamic Car Design [24.258618104493532]
  We introduce the concept of "Design Agents" for engineering applications, particularly focusing on the automotive design process. Our framework integrates AI-driven design agents into the traditional engineering workflow to augment creativity, enhance efficiency, and significantly accelerate the overall design cycle.
  arXiv  Detail & Related papers  (2025-03-30T04:57:17Z)
• AI Automatons: AI Systems Intended to Imitate Humans [54.19152688545896]
  There is a growing proliferation of AI systems designed to mimic people's behavior, work, abilities, likenesses, or humanness. The research, design, deployment, and availability of such AI systems have prompted growing concerns about a wide range of possible legal, ethical, and other social impacts.
  arXiv  Detail & Related papers  (2025-03-04T03:55:38Z)
• Empowering Clients: Transformation of Design Processes Due to Generative AI [1.4003044924094596]
  The study reveals that AI can disrupt the ideation phase by enabling clients to engage in the design process through rapid visualization of their own ideas. Our study shows that while AI can provide valuable feedback on designs, it might fail to generate such designs, allowing for interesting connections to foundations in computer science. Our study also reveals that there is uncertainty among architects about the interpretative sovereignty of architecture and loss of meaning and identity when AI increasingly takes over authorship in the design process.
  arXiv  Detail & Related papers  (2024-11-22T16:48:15Z)
• Survey of User Interface Design and Interaction Techniques in Generative AI Applications [79.55963742878684]
  We aim to create a compendium of different user-interaction patterns that can be used as a reference for designers and developers alike. We also strive to lower the entry barrier for those attempting to learn more about the design of generative AI applications.
  arXiv  Detail & Related papers  (2024-10-28T23:10:06Z)
• Data Analysis in the Era of Generative AI [56.44807642944589]
  This paper explores the potential of AI-powered tools to reshape data analysis, focusing on design considerations and challenges. We explore how the emergence of large language and multimodal models offers new opportunities to enhance various stages of data analysis workflow. We then examine human-centered design principles that facilitate intuitive interactions, build user trust, and streamline the AI-assisted analysis workflow across multiple apps.
  arXiv  Detail & Related papers  (2024-09-27T06:31:03Z)
• MetaDesigner: Advancing Artistic Typography through AI-Driven, User-Centric, and Multilingual WordArt Synthesis [65.78359025027457]
  MetaDesigner revolutionizes artistic typography by leveraging the strengths of Large Language Models (LLMs) to drive a design paradigm centered around user engagement. A comprehensive feedback mechanism harnesses insights from multimodal models and user evaluations to refine and enhance the design process iteratively. Empirical validations highlight MetaDesigner's capability to effectively serve diverse WordArt applications, consistently producing aesthetically appealing and context-sensitive results.
  arXiv  Detail & Related papers  (2024-06-28T11:58:26Z)
• Inspired by AI? A Novel Generative AI System To Assist Conceptual Automotive Design [6.001793288867721]
  Design inspiration is crucial for establishing the direction of a design as well as evoking feelings and conveying meanings during the conceptual design process. Many practice designers use text-based searches on platforms like Pinterest to gather image ideas, followed by sketching on paper or using digital tools to develop concepts. Emerging generative AI techniques, such as diffusion models, offer a promising avenue to streamline these processes by swiftly generating design concepts based on text and image inspiration inputs.
  arXiv  Detail & Related papers  (2024-06-06T17:04:14Z)
• Responding to Generative AI Technologies with Research-through-Design: The Ryelands AI Lab as an Exploratory Study [6.028558240668647]
  We partner with a primary school to develop a constructionist curriculum centered on students interacting with a generative AI technology. We provide a detailed account of the design of and outputs from the curriculum and learning materials, finding centrally that the reflexive and prolonged hands-on' approach led to a co-development of students' practical and critical competencies.
  arXiv  Detail & Related papers  (2024-05-07T21:34:10Z)
• Geometric Deep Learning for Computer-Aided Design: A Survey [85.79012726689511]
  This survey offers a comprehensive overview of learning-based methods in computer-aided design. It includes similarity analysis and retrieval, 2D and 3D CAD model synthesis, and CAD generation from point clouds. It provides a complete list of benchmark datasets and their characteristics, along with open-source codes that have propelled research in this domain.
  arXiv  Detail & Related papers  (2024-02-27T17:11:35Z)
• An Interactive Agent Foundation Model [49.77861810045509]
  We propose an Interactive Agent Foundation Model that uses a novel multi-task agent training paradigm for training AI agents. Our training paradigm unifies diverse pre-training strategies, including visual masked auto-encoders, language modeling, and next-action prediction. We demonstrate the performance of our framework across three separate domains -- Robotics, Gaming AI, and Healthcare.
  arXiv  Detail & Related papers  (2024-02-08T18:58:02Z)
• Design Principles for Generative AI Applications [22.587972924039992]
  Generative AI applications present unique design challenges. There is an urgent need for guidance on how to design user experiences that foster effective and safe use. We present six principles for the design of generative AI applications.
  arXiv  Detail & Related papers  (2024-01-25T19:38:21Z)
• DesignGPT: Multi-Agent Collaboration in Design [4.6272626111555955]
  DesignGPT uses artificial intelligence agents to simulate the roles of different positions in the design company and allows human designers to collaborate with them in natural language. Experimental results show that compared with separate AI tools, DesignGPT improves the performance of designers.
  arXiv  Detail & Related papers  (2023-11-20T08:05:52Z)
• fAIlureNotes: Supporting Designers in Understanding the Limits of AI Models for Computer Vision Tasks [32.53515595703429]
  fAIlureNotes is a designer-centered failure exploration and analysis tool. It supports designers in evaluating models and identifying failures across diverse user groups and scenarios.
  arXiv  Detail & Related papers  (2023-02-22T23:41:36Z)
• Design Space Exploration and Explanation via Conditional Variational Autoencoders in Meta-model-based Conceptual Design of Pedestrian Bridges [52.77024349608834]
  This paper provides a performance-driven design exploration framework to augment the human designer through a Conditional Variational Autoencoder (CVAE) The CVAE is trained on 18'000 synthetically generated instances of a pedestrian bridge in Switzerland.
  arXiv  Detail & Related papers  (2022-11-29T17:28:31Z)
• Investigating Positive and Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniques [55.492211642128446]
  Designers reportedly struggle with design optimization tasks where they are asked to find a combination of design parameters that maximizes a given set of objectives. Model-based computational design algorithms assist designers by generating design examples during design. Black box methods for assistance, on the other hand, can work with any design problem.
  arXiv  Detail & Related papers  (2022-04-15T20:40:43Z)
• DIME: Fine-grained Interpretations of Multimodal Models via Disentangled Local Explanations [119.1953397679783]
  We focus on advancing the state-of-the-art in interpreting multimodal models. Our proposed approach, DIME, enables accurate and fine-grained analysis of multimodal models.
  arXiv  Detail & Related papers  (2022-03-03T20:52:47Z)
• Towards A Process Model for Co-Creating AI Experiences [16.767362787750418]
  Thinking of technology as a design material is appealing to designers. As a material, AI resists this approach because its properties emerge as part of the design process itself. We investigate the co-creation process through a design study with 10 pairs of designers and engineers.
  arXiv  Detail & Related papers  (2021-04-15T16:53:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.