"Like a Nesting Doll": Analyzing Recursion Analogies Generated by CS Students using Large Language Models

• URL: http://arxiv.org/abs/2403.09409v1
• Date: Thu, 14 Mar 2024 14:01:26 GMT
• Title: "Like a Nesting Doll": Analyzing Recursion Analogies Generated by CS Students using Large Language Models
• Authors: Seth Bernstein, Paul Denny, Juho Leinonen, Lauren Kan, Arto Hellas, Matt Littlefield Sami Sarsa, Stephen MacNeil,
• Abstract summary: A good analogy can bridge the gap between unfamiliar concepts and familiar ones, providing an engaging way to aid understanding. We investigate what extent large language models (LLMs), specifically ChatGPT, can provide access to personally relevant analogies on demand.
• Score: 5.162225137921625
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Grasping complex computing concepts often poses a challenge for students who struggle to anchor these new ideas to familiar experiences and understandings. To help with this, a good analogy can bridge the gap between unfamiliar concepts and familiar ones, providing an engaging way to aid understanding. However, creating effective educational analogies is difficult even for experienced instructors. We investigate to what extent large language models (LLMs), specifically ChatGPT, can provide access to personally relevant analogies on demand. Focusing on recursion, a challenging threshold concept, we conducted an investigation analyzing the analogies generated by more than 350 first-year computing students. They were provided with a code snippet and tasked to generate their own recursion-based analogies using ChatGPT, optionally including personally relevant topics in their prompts. We observed a great deal of diversity in the analogies produced with student-prescribed topics, in contrast to the otherwise generic analogies, highlighting the value of student creativity when working with LLMs. Not only did students enjoy the activity and report an improved understanding of recursion, but they described more easily remembering analogies that were personally and culturally relevant.

Related papers

• Boosting Scientific Concepts Understanding: Can Analogy from Teacher Models Empower Student Models? [12.164449420121189]
  We investigate how analogies created by teacher language models (LMs) can assist student LMs in understanding scientific concepts. Our results suggest that free-form analogies can indeed aid LMs in understanding concepts.
  arXiv  Detail & Related papers  (2024-06-17T09:51:38Z)
• AnaloBench: Benchmarking the Identification of Abstract and Long-context Analogies [19.613777134600408]
  Analogical thinking allows humans to solve problems in creative ways. Can language models (LMs) do the same? benchmarking approach focuses on aspects of this ability that are common among humans.
  arXiv  Detail & Related papers  (2024-02-19T18:56:44Z)
• Thread of Thought Unraveling Chaotic Contexts [133.24935874034782]
  "Thread of Thought" (ThoT) strategy draws inspiration from human cognitive processes. In experiments, ThoT significantly improves reasoning performance compared to other prompting techniques.
  arXiv  Detail & Related papers  (2023-11-15T06:54:44Z)
• StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understanding [72.38872974837462]
  We evaluate the ability to identify and generate analogies by constructing a first-of-its-kind large-scale story-level analogy corpus. textscStory Analogy contains 24K story pairs from diverse domains with human annotations on two similarities from the extended Structure-Mapping Theory. We observe that the data in textscStory Analogy can improve the quality of analogy generation in large language models.
  arXiv  Detail & Related papers  (2023-10-19T16:29:23Z)
• Large Language Models as Analogical Reasoners [155.9617224350088]
  Chain-of-thought (CoT) prompting for language models demonstrates impressive performance across reasoning tasks. We introduce a new prompting approach, analogical prompting, designed to automatically guide the reasoning process of large language models.
  arXiv  Detail & Related papers  (2023-10-03T00:57:26Z)
• ARN: Analogical Reasoning on Narratives [13.707344123755126]
  We develop a framework that operationalizes dominant theories of analogy, using narrative elements to create surface and system mappings. We show that while all LLMs can largely recognize near analogies, even the largest ones struggle with far analogies in a zero-shot setting.
  arXiv  Detail & Related papers  (2023-10-02T08:58:29Z)
• Why Do We Need Neuro-symbolic AI to Model Pragmatic Analogies? [6.8107181513711055]
  A hallmark of intelligence is the ability to use a familiar domain to make inferences about a less familiar domain, known as analogical reasoning. We discuss analogies at four distinct levels of complexity: lexical analogies, syntactic analogies, semantic analogies, and pragmatic analogies. We employ Neuro-symbolic AI techniques that combine statistical and symbolic AI, informing the representation of unstructured text to highlight and augment relevant content, provide abstraction and guide the mapping process.
  arXiv  Detail & Related papers  (2023-08-02T21:13:38Z)
• Beneath Surface Similarity: Large Language Models Make Reasonable Scientific Analogies after Structure Abduction [46.2032673640788]
  The vital role of analogical reasoning in human cognition allows us to grasp novel concepts by linking them with familiar ones through shared relational structures. This work suggests that Large Language Models (LLMs) often overlook the structures that underpin these analogies. This paper introduces a task of analogical structure abduction, grounded in cognitive psychology, designed to abduce structures that form an analogy between two systems.
  arXiv  Detail & Related papers  (2023-05-22T03:04:06Z)
• Analogy-Making as a Core Primitive in the Software Engineering Toolbox [7.396342576390398]
  We argue that analogy making should be seen as a core primitive in software engineering. We demonstrate this idea using Sifter, a new analogy-making algorithm suitable for software engineering applications.
  arXiv  Detail & Related papers  (2020-09-14T17:24:15Z)
• Few-shot Visual Reasoning with Meta-analogical Contrastive Learning [141.2562447971]
  We propose to solve a few-shot (or low-shot) visual reasoning problem, by resorting to analogical reasoning. We extract structural relationships between elements in both domains, and enforce them to be as similar as possible with analogical learning. We validate our method on RAVEN dataset, on which it outperforms state-of-the-art method, with larger gains when the training data is scarce.
  arXiv  Detail & Related papers  (2020-07-23T14:00:34Z)
• A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation [98.25464306634758]
  We propose to utilize commonsense knowledge from external knowledge bases to generate reasonable stories. We employ multi-task learning which combines a discriminative objective to distinguish true and fake stories. Our model can generate more reasonable stories than state-of-the-art baselines, particularly in terms of logic and global coherence.
  arXiv  Detail & Related papers  (2020-01-15T05:42:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.