From Answer Givers to Design Mentors: Guiding LLMs with the Cognitive Apprenticeship Model

• URL: http://arxiv.org/abs/2601.19053v1
• Date: Tue, 27 Jan 2026 00:27:15 GMT
• Title: From Answer Givers to Design Mentors: Guiding LLMs with the Cognitive Apprenticeship Model
• Authors: Yongsu Ahn, Lejun R Liao, Benjamin Bach, Nam Wook Kim,
• Abstract summary: Large Language Models (LLMs) can support design work, but often provide generic, one-off suggestions that limit reflective engagement.<n>We apply the Cognitive Apprenticeship Model, which emphasizes demonstrating reasoning through six methods.<n>We operationalize these instructional methods through structured prompting and evaluate them in a within-subjects study with data visualization practitioners.
• Score: 15.804452611205454
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Design feedback helps practitioners improve their artifacts while also fostering reflection and design reasoning. Large Language Models (LLMs) such as ChatGPT can support design work, but often provide generic, one-off suggestions that limit reflective engagement. We investigate how to guide LLMs to act as design mentors by applying the Cognitive Apprenticeship Model, which emphasizes demonstrating reasoning through six methods: modeling, coaching, scaffolding, articulation, reflection, and exploration. We operationalize these instructional methods through structured prompting and evaluate them in a within-subjects study with data visualization practitioners. Participants interacted with both a baseline LLM and an instructional LLM designed with cognitive apprenticeship prompts. Surveys, interviews, and conversational log analyses compared experiences across conditions. Our findings show that cognitively informed prompts elicit deeper design reasoning and more reflective feedback exchanges, though the baseline is sometimes preferred depending on task types or experience levels. We distill design considerations for AI-assisted feedback systems that foster reflective practice.

Related papers

• Learning from Trials and Errors: Reflective Test-Time Planning for Embodied LLMs [63.88783817420284]
  Embodied robots cannot reflect on what went wrong or why, turning deployment into a sequence of independent trials.<n>We introduce Reflective Test-Time Planning, which integrates two modes of reflection: textitreflection-in-action and textitreflection-on-action<n>We also include retrospective reflection, allowing the agent to re-evaluate earlier decisions and perform model updates with hindsight.
  arXiv  Detail & Related papers  (2026-02-24T18:55:18Z)
• Owlgorithm: Supporting Self-Regulated Learning in Competitive Programming through LLM-Driven Reflection [0.0]
  We present an educational platform that supports Self-Regulated Learning (SRL) in competitive programming (CP)<n>Owlgorithm produces context-aware, meta prompts tailored to individual student submissions.<n>Our exploratory assessment of student ratings and TA feedback revealed both promising benefits and notable limitations.
  arXiv  Detail & Related papers  (2025-11-13T05:08:45Z)
• Listening with Language Models: Using LLMs to Collect and Interpret Classroom Feedback [14.83267437400996]
  Large Language Model (LLM)-powered chatbots can reimagine the classroom feedback process by engaging students in reflective, conversational dialogues.<n>Our findings suggest that LLM-based feedback systems offer richer insights, greater contextual relevance, and higher engagement compared to standard survey tools.
  arXiv  Detail & Related papers  (2025-08-13T22:53:55Z)
• \ extsc{SimInstruct}: A Responsible Tool for Collecting Scaffolding Dialogues Between Experts and LLM-Simulated Novices [21.67295740032255]
  SimInstruct is a scalable, expert-in-the-loop tool for collecting scaffolding dialogues.<n>Using teaching development coaching as an example domain, SimInstruct simulates novice instructors via LLMs.<n>Our results reveal that persona traits, such as extroversion and introversion, meaningfully influence how experts engage.
  arXiv  Detail & Related papers  (2025-08-06T13:16:10Z)
• Automated Feedback on Student-Generated UML and ER Diagrams Using Large Language Models [39.58317527488534]
  We introduce DUET (Diamatic & ER Tutor), a prototype of an LLM-based tool.<n>It converts a reference diagram and a student-submitted diagram into a textual representation and provides structured feedback based on the differences.<n>It uses a multi-stage LLM pipeline to compare diagrams and generate reflective feedback.<n>It enables analytical insights for educators, aiming to foster self-directed learning and inform instructional strategies.
  arXiv  Detail & Related papers  (2025-07-31T11:49:01Z)
• Personalized Exercise Recommendation with Semantically-Grounded Knowledge Tracing [54.44838681588145]
  ExRec is a framework for personalized exercise recommendation with semantically-grounded knowledge tracing.<n>We show that ExRec generalizes robustly to new, unseen questions and that it produces interpretable student learning trajectories.
  arXiv  Detail & Related papers  (2025-07-15T07:54:04Z)
• Meta-Reflection: A Feedback-Free Reflection Learning Framework [57.14485943991588]
  We propose Meta-Reflection, a feedback-free reflection mechanism that requires only a single inference pass without external feedback.<n>Motivated by the human ability to remember and retrieve reflections from past experiences, Meta-Reflection integrates reflective insights into a codebook.<n>To thoroughly investigate and evaluate the practicality of Meta-Reflection in real-world scenarios, we introduce an industrial e-commerce benchmark named E-commerce Customer Intent Detection.
  arXiv  Detail & Related papers  (2024-12-18T12:20:04Z)
• Cognitive LLMs: Towards Integrating Cognitive Architectures and Large Language Models for Manufacturing Decision-making [51.737762570776006]
  LLM-ACTR is a novel neuro-symbolic architecture that provides human-aligned and versatile decision-making. Our framework extracts and embeds knowledge of ACT-R's internal decision-making process as latent neural representations. Our experiments on novel Design for Manufacturing tasks show both improved task performance as well as improved grounded decision-making capability.
  arXiv  Detail & Related papers  (2024-08-17T11:49:53Z)
• Enhancing LLM-Based Feedback: Insights from Intelligent Tutoring Systems and the Learning Sciences [0.0]
  This work advocates careful and caring AIED research by going through previous research on feedback generation in ITS. The main contributions of this paper include: an avocation of applying more cautious, theoretically grounded methods in feedback generation in the era of generative AI.
  arXiv  Detail & Related papers  (2024-05-07T20:09:18Z)
• ProSwitch: Knowledge-Guided Instruction Tuning to Switch Between Professional and Non-Professional Responses [56.949741308866535]
  Large Language Models (LLMs) have demonstrated efficacy in various linguistic applications.<n>This study introduces a novel approach, named ProSwitch, which enables a language model to switch between professional and non-professional answers.
  arXiv  Detail & Related papers  (2024-03-14T06:49:16Z)
• From Language Modeling to Instruction Following: Understanding the Behavior Shift in LLMs after Instruction Tuning [63.63840740526497]
  We investigate how instruction tuning adjusts pre-trained models with a focus on intrinsic changes. The impact of instruction tuning is then studied by comparing the explanations derived from the pre-trained and instruction-tuned models. Our findings reveal three significant impacts of instruction tuning.
  arXiv  Detail & Related papers  (2023-09-30T21:16:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.