Prober.ai: Gated Inquiry-Based Feedback via LLM-Constrained Personas for Argumentative Writing Development

• URL: http://arxiv.org/abs/2605.05598v1
• Date: Thu, 07 May 2026 02:32:01 GMT
• Title: Prober.ai: Gated Inquiry-Based Feedback via LLM-Constrained Personas for Argumentative Writing Development
• Abstract summary: Prober.ai is a web-based writing environment that inverts the conventional AI-tutoring paradigm.<n>Rather than generating or rewriting student text, the system constrains an LLM through persona-specific system prompts.
• Score: 0.5352419666794658
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The proliferation of large language models (LLMs) in educational settings has paradoxically undermined the cognitive processes they purport to support. Students increasingly outsource critical thinking to AI assistants that generate polished text on demand, resulting in measurable cognitive debt and diminished argumentative reasoning skills. We present Prober.ai, a web-based writing environment that inverts the conventional AI-tutoring paradigm: rather than generating or rewriting student text, the system constrains an LLM (Gemini 3 Flash Preview) through persona-specific system prompts and structured JSON output schemas to produce only targeted, inquiry-based questions about argumentative weaknesses. A two-phase interaction architecture -- Challenge and Unlock -- implements a pedagogical friction mechanism whereby revision suggestions are gated behind mandatory student reflection. The system's design is grounded in Toulmin's argumentation theory, research on peer feedforward questioning mechanisms, and evidence on AI-supported feedback in writing instruction. A functional prototype was developed in 36 hours during the NY EdTech Hackathon (March 2026), where it was awarded second place. We describe the system architecture, the prompt engineering methodology for constraining LLM output to pedagogically aligned JSON schemas, and discuss implications for scalable, cognition-preserving AI integration in writing education.

Related papers

• Beyond AI Delegation: A Prompt Pattern Framework for Productive Struggle and Evaluative Judgement in Secure Coding Education [0.45706494756895794]
  Large language models make it easy for students to delegate writing, analysis, and problem-solving to automated systems.<n>We introduce a practical framework that helps educators keep GenAI in the course without removing the cognitive demands that make it worthwhile.
  arXiv  Detail & Related papers  (2026-05-23T07:35:49Z)
• Interpretable Difficulty-Aware Knowledge Tracing in Tutor-Student Dialogues [46.915280360954576]
  We propose an interpretable difficulty-aware conversational KT framework built upon large language models (LLMs)<n>The framework incorporates the original textual question and the next tutor-posed task to estimate the student's knowledge state and the difficulty of the upcoming turn.<n>We evaluate the framework on two tutor-student dialogue datasets.
  arXiv  Detail & Related papers  (2026-05-01T21:07:24Z)
• Chatbot-Based Assessment of Code Understanding in Automated Programming Assessment Systems [1.0705399532413615]
  This paper reports a saturation-based scoping review of conversational assessment approaches in programming education.<n>The review identifies three dominant architectural families: rule-based or template-driven systems, LLM-based systems, and hybrid systems.<n>The paper synthesizes these findings into a Hybrid Socratic Framework for integrating conversational verification into Automated Programming Assessment Systems.
  arXiv  Detail & Related papers  (2026-04-08T17:11:30Z)
• From Use to Oversight: How Mental Models Influence User Behavior and Output in AI Writing Assistants [70.1699066541501]
  We examine two types of mental models -- functional or related to what the system does, and structural or related to how the system works.<n>We find that while participants in the structural mental model condition demonstrate a better understanding of the system, this can have a backfiring effect.
  arXiv  Detail & Related papers  (2026-04-06T20:50:44Z)
• Student Perceptions of Large Language Models Use in Self-Reflection and Design Critique in Architecture Studio [0.0]
  This study investigates the integration of Large Language Models (LLMs) into the feedback mechanisms of the architectural design studio.<n>The research analyzes student perceptions across three distinct feed-back domains: self-reflection, peer critique, and professor-led reviews.
  arXiv  Detail & Related papers  (2026-01-19T06:38:03Z)
• Improving Procedural Skill Explanations via Constrained Generation: A Symbolic-LLM Hybrid Architecture [2.6141337419964725]
  In procedural skill learning, instructional explanations must convey not just steps, but the causal, goal-directed, and compositional logic behind them.<n>We present Ivy, an AI coaching system that delivers structured, multi-step explanations by combining symbolic Task-Method-Knowledge (TMK) models with a generative interpretation layer-an generative interpretation layer-that constructs explanations while being constrained by TMK structure.
  arXiv  Detail & Related papers  (2025-11-26T00:29:53Z)
• A vibe coding learning design to enhance EFL students' talking to, through, and about AI [5.671810777243196]
  This article reports on the piloting of vibe coding (using natural language to create software applications with AI) for English as a Foreign Language education.<n>We developed a human-AI meta-languaging framework with three dimensions: talking to AI (prompt engineering), talking through AI (negotiating authorship), and talking about AI.
  arXiv  Detail & Related papers  (2025-09-09T00:27:04Z)
• NeuralNexus at BEA 2025 Shared Task: Retrieval-Augmented Prompting for Mistake Identification in AI Tutors [0.12499537119440242]
  This paper presents our system for Track 1, Mistake Identification in the BEA 2025 Shared Task on Pedagogical Ability Assessment of AI-powered Tutors.<n>The task involves evaluating whether a tutor's response correctly identifies a mistake in a student's reasoning.<n>Our system retrieves semantically similar examples, constructs structured prompts, and uses schema-guided parsing produceable predictions.
  arXiv  Detail & Related papers  (2025-06-12T12:11:56Z)
• Language Models can be Logical Solvers [99.40649402395725]
  We introduce LoGiPT, a novel language model that directly emulates the reasoning processes of logical solvers. LoGiPT is fine-tuned on a newly constructed instruction-tuning dataset derived from revealing and refining the invisible reasoning process of deductive solvers.
  arXiv  Detail & Related papers  (2023-11-10T16:23:50Z)
• DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text [73.68051228972024]
  Large Language Models (LLMs) have exhibited impressive generation capabilities, but they suffer from hallucinations when relying on their internal knowledge. Retrieval-augmented LLMs have emerged as a potential solution to ground LLMs in external knowledge.
  arXiv  Detail & Related papers  (2023-10-31T04:37:57Z)
• Re-Reading Improves Reasoning in Large Language Models [87.46256176508376]
  We introduce a simple, yet general and effective prompting method, Re2, to enhance the reasoning capabilities of off-the-shelf Large Language Models (LLMs) Unlike most thought-eliciting prompting methods, such as Chain-of-Thought (CoT), Re2 shifts the focus to the input by processing questions twice, thereby enhancing the understanding process. We evaluate Re2 on extensive reasoning benchmarks across 14 datasets, spanning 112 experiments, to validate its effectiveness and generality.
  arXiv  Detail & Related papers  (2023-09-12T14:36:23Z)
• Investigating Fairness Disparities in Peer Review: A Language Model Enhanced Approach [77.61131357420201]
  We conduct a thorough and rigorous study on fairness disparities in peer review with the help of large language models (LMs) We collect, assemble, and maintain a comprehensive relational database for the International Conference on Learning Representations (ICLR) conference from 2017 to date. We postulate and study fairness disparities on multiple protective attributes of interest, including author gender, geography, author, and institutional prestige.
  arXiv  Detail & Related papers  (2022-11-07T16:19:42Z)
• A general framework for scientifically inspired explanations in AI [76.48625630211943]
  We instantiate the concept of structure of scientific explanation as the theoretical underpinning for a general framework in which explanations for AI systems can be implemented. This framework aims to provide the tools to build a "mental-model" of any AI system so that the interaction with the user can provide information on demand and be closer to the nature of human-made explanations.
  arXiv  Detail & Related papers  (2020-03-02T10:32:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.