Tracing the Invisible: Understanding Students' Judgment in AI-Supported Design Work

• URL: http://arxiv.org/abs/2505.08939v1
• Date: Tue, 13 May 2025 20:08:10 GMT
• Title: Tracing the Invisible: Understanding Students' Judgment in AI-Supported Design Work
• Authors: Suchismita Naik, Prakash Shukla, Ike Obi, Jessica Backus, Nancy Rasche, Paul Parsons,
• Abstract summary: This study analyzes reflections from 33 student teams in an HCI design course to examine the kinds of judgments students make when using AI tools.<n>We found both established forms of design judgment (e.g., instrumental, appreciative, quality) and emergent types: agency-distribution judgment and reliability judgment.<n>Our findings suggest that generative AI introduces new layers of complexity into design reasoning, prompting students to reflect on how and when to rely on it.
• Score: 2.8038082486377114
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As generative AI tools become integrated into design workflows, students increasingly engage with these tools not just as aids, but as collaborators. This study analyzes reflections from 33 student teams in an HCI design course to examine the kinds of judgments students make when using AI tools. We found both established forms of design judgment (e.g., instrumental, appreciative, quality) and emergent types: agency-distribution judgment and reliability judgment. These new forms capture how students negotiate creative responsibility with AI and assess the trustworthiness of its outputs. Our findings suggest that generative AI introduces new layers of complexity into design reasoning, prompting students to reflect not only on what AI produces, but also on how and when to rely on it. By foregrounding these judgments, we offer a conceptual lens for understanding how students engage in co-creative sensemaking with AI in design contexts.

Related papers

• Humanizing AI Grading: Student-Centered Insights on Fairness, Trust, Consistency and Transparency [0.6138671548064355]
  This study investigates students' perceptions of Artificial Intelligence (AI) grading systems in an undergraduate computer science course.<n>Findings reveal concerns about AI's lack of contextual understanding and personalization.<n>This work contributes to ethics-centered assessment practices by amplifying student voices and offering design principles for humanizing AI in designed learning environments.
  arXiv  Detail & Related papers  (2026-02-08T01:18:10Z)
• From Individual Prompts to Collective Intelligence: Mainstreaming Generative AI in the Classroom [2.7886031629181987]
  We argue for a shift toward collective intelligence (CI)-focused pedagogy, where GenAI acts as a catalyst for peer-to-peer learning.<n>We implement Generative CI activities in two undergraduate engineering courses, engaging 140 students through thinking routines.<n>Results demonstrate that students value the combination of human collaboration with strategic AI support.
  arXiv  Detail & Related papers  (2026-01-07T08:28:20Z)
• To Use or to Refuse? Re-Centering Student Agency with Generative AI in Engineering Design Education [14.581313362006272]
  This pilot study traces students' reflections on the use of AI in a 13-week foundational design course enrolling over 500 first-year engineering and architecture students at the Singapore University of Technology and Design.<n>Students were required to reflect on whether the technology was used as a tool (instrumental assistant), a teammate (collaborative partner), or neither.<n>Students learned to use AI for innovation rather than just automation and to reflect on agency, ethics, and context rather than on prompt crafting alone.
  arXiv  Detail & Related papers  (2025-10-22T08:06:48Z)
• AI in Design Education at College Level-Educators' Perspectives and Challenges [0.0]
  This study examines how current design educators perceive the role of AI in college-level design education.<n>The findings reveal that AI, as a tool and source of information, has become an integral part of design education.<n>At the same time, educators express anticipation for advancements in ethical standards, authenticity, and the resolution of copyright issues related to AI.
  arXiv  Detail & Related papers  (2025-07-23T13:02:37Z)
• Beyond Detection: Designing AI-Resilient Assessments with Automated Feedback Tool to Foster Critical Thinking [0.0]
  This research proposes a proactive, AI-resilient solution based on assessment design rather than detection.<n>It introduces a web-based Python tool that integrates Bloom's taxonomy with advanced natural language processing techniques.<n>It helps educators determine whether a task targets lower-order thinking such as recall and summarization or higher-order skills such as analysis, evaluation, and creation.
  arXiv  Detail & Related papers  (2025-03-30T23:13:00Z)
• AI Automatons: AI Systems Intended to Imitate Humans [54.19152688545896]
  There is a growing proliferation of AI systems designed to mimic people's behavior, work, abilities, likenesses, or humanness.<n>The research, design, deployment, and availability of such AI systems have prompted growing concerns about a wide range of possible legal, ethical, and other social impacts.
  arXiv  Detail & Related papers  (2025-03-04T03:55:38Z)
• Scaffolding Creativity: Integrating Generative AI Tools and Real-world Experiences in Business Education [0.0]
  This study investigates the intersection of Generative AI tools and experiential learning in business education.<n>Through a case study of an innovative undergraduate course, we examine how students interact with and adapt to various AI modalities.<n>Our findings reveal how this integrated approach enables novice users to overcome creative barriers, accelerates skill acquisition, and creates a dynamic interplay between AI-generated insights and real-world validation.
  arXiv  Detail & Related papers  (2025-01-11T12:31:10Z)
• Understanding Student Acceptance, Trust, and Attitudes Toward AI-Generated Images for Educational Purposes [1.0878040851637998]
  This study assesses students' acceptance, trust, and positive attitudes towards AI-generated images for educational tasks. The results reveal high acceptance, trust, and positive attitudes among students who value the ease of use and potential academic benefits. These findings suggest a need for developing comprehensive guidelines that address ethical considerations and intellectual property issues.
  arXiv  Detail & Related papers  (2024-11-24T04:39:48Z)
• Combining AI Control Systems and Human Decision Support via Robustness and Criticality [53.10194953873209]
  We extend a methodology for adversarial explanations (AE) to state-of-the-art reinforcement learning frameworks. We show that the learned AI control system demonstrates robustness against adversarial tampering. In a training / learning framework, this technology can improve both the AI's decisions and explanations through human interaction.
  arXiv  Detail & Related papers  (2024-07-03T15:38:57Z)
• Charting the Future of AI in Project-Based Learning: A Co-Design Exploration with Students [35.05435052195561]
  The increasing use of Artificial Intelligence (AI) by students in learning presents new challenges for assessing their learning outcomes. This paper introduces a co-design study to explore the potential of students' AI usage data as a novel material for assessment.
  arXiv  Detail & Related papers  (2024-01-26T14:49:29Z)
• Grasping AI: experiential exercises for designers [8.95562850825636]
  We investigate techniques for exploring and reflecting on the interactional affordances, the unique relational possibilities, and the wider social implications of AI systems. We find that exercises around metaphors and enactments make questions of training and learning, privacy and consent, autonomy and agency more tangible.
  arXiv  Detail & Related papers  (2023-10-02T15:34:08Z)
• Assigning AI: Seven Approaches for Students, with Prompts [0.0]
  This paper examines the transformative role of Large Language Models (LLMs) in education and their potential as learning tools. The authors propose seven approaches for utilizing AI in classrooms: AI-tutor, AI-coach, AI-mentor, AI-teammate, AI-tool, AI-simulator, and AI-student.
  arXiv  Detail & Related papers  (2023-06-13T03:36:36Z)
• Designerly Understanding: Information Needs for Model Transparency to Support Design Ideation for AI-Powered User Experience [42.73738624139124]
  Designers face hurdles understanding AI technologies, such as pre-trained language models, as design materials. This limits their ability to ideate and make decisions about whether, where, and how to use AI. Our study highlights the pivotal role that UX designers can play in Responsible AI.
  arXiv  Detail & Related papers  (2023-02-21T02:06:24Z)
• Seamful XAI: Operationalizing Seamful Design in Explainable AI [59.89011292395202]
  Mistakes in AI systems are inevitable, arising from both technical limitations and sociotechnical gaps. We propose that seamful design can foster AI explainability by revealing sociotechnical and infrastructural mismatches. We explore this process with 43 AI practitioners and real end-users.
  arXiv  Detail & Related papers  (2022-11-12T21:54:05Z)
• Stakeholder Participation in AI: Beyond "Add Diverse Stakeholders and Stir" [76.44130385507894]
  This paper aims to ground what we dub a 'participatory turn' in AI design by synthesizing existing literature on participation and through empirical analysis of its current practices. Based on our literature synthesis and empirical research, this paper presents a conceptual framework for analyzing participatory approaches to AI design.
  arXiv  Detail & Related papers  (2021-11-01T17:57:04Z)
• Building Bridges: Generative Artworks to Explore AI Ethics [56.058588908294446]
  In recent years, there has been an increased emphasis on understanding and mitigating adverse impacts of artificial intelligence (AI) technologies on society. A significant challenge in the design of ethical AI systems is that there are multiple stakeholders in the AI pipeline, each with their own set of constraints and interests. This position paper outlines some potential ways in which generative artworks can play this role by serving as accessible and powerful educational tools.
  arXiv  Detail & Related papers  (2021-06-25T22:31:55Z)
• An interdisciplinary conceptual study of Artificial Intelligence (AI) for helping benefit-risk assessment practices: Towards a comprehensive qualification matrix of AI programs and devices (pre-print 2020) [55.41644538483948]
  This paper proposes a comprehensive analysis of existing concepts coming from different disciplines tackling the notion of intelligence. The aim is to identify shared notions or discrepancies to consider for qualifying AI systems.
  arXiv  Detail & Related papers  (2021-05-07T12:01:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.