Design Generative AI for Practitioners: Exploring Interaction Approaches Aligned with Creative Practice

• URL: http://arxiv.org/abs/2603.03074v1
• Date: Tue, 03 Mar 2026 15:19:23 GMT
• Title: Design Generative AI for Practitioners: Exploring Interaction Approaches Aligned with Creative Practice
• Authors: Xiaohan Peng, Wendy E. Mackay, Janin Koch,
• Abstract summary: We present three interaction approaches that distribute control across intent, input, and process.<n>We argue that alignment is a dynamic negotiation, with AI adopting proactive or reactive roles according to designers' instrumental and inspirational needs and the creative stage.
• Score: 20.726284013271584
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Design is a non-linear, reflective process in which practitioners engage with visual, semantic, and other expressive materials to explore, iterate, and refine ideas. As Generative AI (GenAI) becomes integrated into professional design practice, traditional interaction approaches focusing on prompts or whole-image manipulation can misalign AI output with designers' intent, forcing visual thinkers into verbal reasoning or post-hoc adjustments. We present three interaction approaches from DesignPrompt, FusAIn, and DesignTrace that distribute control across intent, input, and process, enabling designers to guide AI alignment at different stages of interaction. We further argue that alignment is a dynamic negotiation, with AI adopting proactive or reactive roles according to designers' instrumental and inspirational needs and the creative stage.

Related papers

• Synergizing Understanding and Generation with Interleaved Analyzing-Drafting Thinking [154.2388970262703]
  Unified Vision-Language Models (UVLMs) aim to advance multimodal learning by supporting both understanding and generation within a single framework.<n>We introduce the interleaved Analyzing-Drafting problem-solving loop (AD-Loop), a new think paradigm that alternates between analytic and drafting operations.<n>By interleaving textual thoughts with visual thoughts, AD-Loop enables models to iteratively refine both comprehension and outputs, fostering genuine synergy.
  arXiv  Detail & Related papers  (2026-02-24T23:26:09Z)
• Beyond Pixels: Visual Metaphor Transfer via Schema-Driven Agentic Reasoning [56.24016465596292]
  A visual metaphor constitutes a high-order form of human creativity, employing cross-domain semantic fusion to transform abstract concepts into impactful visual rhetoric.<n>We introduce the task of Visual Metaphor Transfer (VMT), which challenges models to autonomously decouple the "creative essence" from a reference image and re-materialize that abstract logic onto a user-specified subject.<n>Our method significantly outperforms SOTA baselines in metaphor consistency, analogy appropriateness, and visual creativity, paving the way for automated high-impact creative applications in advertising and media.
  arXiv  Detail & Related papers  (2026-02-01T17:01:36Z)
• ThematicPlane: Bridging Tacit User Intent and Latent Spaces for Image Generation [49.805992099208595]
  We introduce ThematicPlane, a system that enables users to navigate and manipulate high-level semantic concepts.<n>This interface bridges the gap between tacit creative intent and system control.
  arXiv  Detail & Related papers  (2025-08-08T06:57:14Z)
• Human-AI Co-Creation: A Framework for Collaborative Design in Intelligent Systems [0.6526824510982802]
  This paper explores the emergent paradigm of human-AI co-creation, where AI is not merely used for automation or efficiency gains, but actively participates in ideation, visual conceptualization, and decision-making.<n>Specifically, we investigate the use of large language models (LLMs) like GPT-4 and multimodal diffusion models such as Stable Diffusion as creative agents that engage designers in iterative cycles of proposal, critique, and revision.
  arXiv  Detail & Related papers  (2025-07-22T04:29:33Z)
• Survey of User Interface Design and Interaction Techniques in Generative AI Applications [79.55963742878684]
  We aim to create a compendium of different user-interaction patterns that can be used as a reference for designers and developers alike. We also strive to lower the entry barrier for those attempting to learn more about the design of generative AI applications.
  arXiv  Detail & Related papers  (2024-10-28T23:10:06Z)
• A Novel Idea Generation Tool using a Structured Conversational AI (CAI) System [0.0]
  This paper presents a novel conversational AI-enabled active ideation interface as a creative idea-generation tool to assist novice designers. It is a dynamic, interactive, and contextually responsive approach, actively involving a large language model (LLM) from the domain of natural language processing (NLP) in artificial intelligence (AI) Integrating such AI models with ideation creates what we refer to as an Active Ideation scenario, which helps foster continuous dialogue-based interaction, context-sensitive conversation, and prolific idea generation.
  arXiv  Detail & Related papers  (2024-09-09T16:02:27Z)
• Diffusion-Based Visual Art Creation: A Survey and New Perspectives [51.522935314070416]
  This survey explores the emerging realm of diffusion-based visual art creation, examining its development from both artistic and technical perspectives. Our findings reveal how artistic requirements are transformed into technical challenges and highlight the design and application of diffusion-based methods within visual art creation. We aim to shed light on the mechanisms through which AI systems emulate and possibly, enhance human capacities in artistic perception and creativity.
  arXiv  Detail & Related papers  (2024-08-22T04:49:50Z)
• MetaDesigner: Advancing Artistic Typography Through AI-Driven, User-Centric, and Multilingual WordArt Synthesis [65.78359025027457]
  MetaDesigner introduces a transformative framework for artistic typography, powered by Large Language Models (LLMs)<n>Its foundation is a multi-agent system comprising the Pipeline, Glyph, and Texture agents, which collectively orchestrate the creation of customizable WordArt.
  arXiv  Detail & Related papers  (2024-06-28T11:58:26Z)
• Inspired by AI? A Novel Generative AI System To Assist Conceptual Automotive Design [6.001793288867721]
  Design inspiration is crucial for establishing the direction of a design as well as evoking feelings and conveying meanings during the conceptual design process. Many practice designers use text-based searches on platforms like Pinterest to gather image ideas, followed by sketching on paper or using digital tools to develop concepts. Emerging generative AI techniques, such as diffusion models, offer a promising avenue to streamline these processes by swiftly generating design concepts based on text and image inspiration inputs.
  arXiv  Detail & Related papers  (2024-06-06T17:04:14Z)
• Prototyping with Prompts: Emerging Approaches and Challenges in Generative AI Design for Collaborative Software Teams [2.237039275844699]
  Generative AI models are increasingly being integrated into human task, enabling the production of expressive content.<n>Unlike traditional human-AI design methods, the new approach to designing generative capabilities focuses heavily on prompt engineering strategies.<n>Our findings highlight emerging practices and role shifts in AI system prototyping among multistakeholder teams.
  arXiv  Detail & Related papers  (2024-02-27T17:56:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.