Using machine learning to measure evidence of students' sensemaking in physics courses

• URL: http://arxiv.org/abs/2503.15638v2
• Date: Sun, 13 Apr 2025 19:10:36 GMT
• Title: Using machine learning to measure evidence of students' sensemaking in physics courses
• Authors: Kaitlin Gili, Kyle Heuton, Astha Shah, Michael C. Hughes,
• Abstract summary: In education, problem-solving correctness is often inappropriately conflated with student learning.<n>In this work, we contribute such a measurement scheme, which quantifies the evidence of students' physical sensemaking given their written explanations for their solutions to physics problems.<n>We implement three unique language encoders with logistic regression, and provide a deployability analysis on 385 real student explanations from the 2023 Introduction to Physics course at Tufts University.
• Score: 5.509349550209279
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In the education system, problem-solving correctness is often inappropriately conflated with student learning. Advances in both Physics Education Research (PER) and Machine Learning (ML) provide the initial tools to develop a more meaningful and efficient measurement scheme for whether physics students are engaging in sensemaking: a learning process of figuring out the how and why for a particular phenomena. In this work, we contribute such a measurement scheme, which quantifies the evidence of students' physical sensemaking given their written explanations for their solutions to physics problems. We outline how the proposed human annotation scheme can be automated into a deployable ML model using language encoders and shared probabilistic classifiers. The procedure is scalable for a large number of problems and students. We implement three unique language encoders with logistic regression, and provide a deployability analysis on 385 real student explanations from the 2023 Introduction to Physics course at Tufts University. Furthermore, we compute sensemaking scores for all students, and analyze these measurements alongside their corresponding problem-solving accuracies. We find no linear relationship between these two variables, supporting the hypothesis that one is not a reliable proxy for the other. We discuss how sensemaking scores can be used alongside problem-solving accuracies to provide a more nuanced snapshot of student performance in physics class.

Related papers

• LLM-based Cognitive Models of Students with Misconceptions [55.29525439159345]
  This paper investigates whether Large Language Models (LLMs) can be instruction-tuned to meet this dual requirement. We introduce MalAlgoPy, a novel Python library that generates datasets reflecting authentic student solution patterns. Our insights enhance our understanding of AI-based student models and pave the way for effective adaptive learning systems.
  arXiv  Detail & Related papers  (2024-10-16T06:51:09Z)
• A Personalised Learning Tool for Physics Undergraduate Students Built On a Large Language Model for Symbolic Regression [0.6666419797034796]
  Interleaved practice enhances the memory and problem-solving ability of students in undergraduate courses. We introduce a personalized learning tool built on a Large Language Model (LLM) that can provide immediate and personalized attention to students.
  arXiv  Detail & Related papers  (2024-06-17T13:43:30Z)
• SimLM: Can Language Models Infer Parameters of Physical Systems? [56.38608628187024]
  We investigate the performance of Large Language Models (LLMs) at performing parameter inference in the context of physical systems. Our experiments suggest that they are not inherently suited to this task, even for simple systems. We propose a promising direction of exploration, which involves the use of physical simulators to augment the context of LLMs.
  arXiv  Detail & Related papers  (2023-12-21T12:05:19Z)
• X-VoE: Measuring eXplanatory Violation of Expectation in Physical Events [75.94926117990435]
  This study introduces X-VoE, a benchmark dataset to assess AI agents' grasp of intuitive physics. X-VoE establishes a higher bar for the explanatory capacities of intuitive physics models. We present an explanation-based learning system that captures physics dynamics and infers occluded object states.
  arXiv  Detail & Related papers  (2023-08-21T03:28:23Z)
• Physics-Based Task Generation through Causal Sequence of Physical Interactions [3.2244944291325996]
  Performing tasks in a physical environment is a crucial yet challenging problem for AI systems operating in the real world. We present a systematic approach for defining a physical scenario using a causal sequence of physical interactions between objects. We then propose a methodology for generating tasks in a physics-simulating environment using defined scenarios as inputs.
  arXiv  Detail & Related papers  (2023-08-05T10:15:18Z)
• Towards a Holistic Understanding of Mathematical Questions with Contrastive Pre-training [65.10741459705739]
  We propose a novel contrastive pre-training approach for mathematical question representations, namely QuesCo. We first design two-level question augmentations, including content-level and structure-level, which generate literally diverse question pairs with similar purposes. Then, to fully exploit hierarchical information of knowledge concepts, we propose a knowledge hierarchy-aware rank strategy.
  arXiv  Detail & Related papers  (2023-01-18T14:23:29Z)
• Lila: A Unified Benchmark for Mathematical Reasoning [59.97570380432861]
  LILA is a unified mathematical reasoning benchmark consisting of 23 diverse tasks along four dimensions. We construct our benchmark by extending 20 datasets benchmark by collecting task instructions and solutions in the form of Python programs. We introduce BHASKARA, a general-purpose mathematical reasoning model trained on LILA.
  arXiv  Detail & Related papers  (2022-10-31T17:41:26Z)
• Symmetry Group Equivariant Architectures for Physics [52.784926970374556]
  In the domain of machine learning, an awareness of symmetries has driven impressive performance breakthroughs. We argue that both the physics community and the broader machine learning community have much to understand.
  arXiv  Detail & Related papers  (2022-03-11T18:27:04Z)
• Physics-informed Reinforcement Learning for Perception and Reasoning about Fluids [0.0]
  We propose a physics-informed reinforcement learning strategy for fluid perception and reasoning from observations. We develop a method for the tracking (perception) and analysis (reasoning) of any previously unseen liquid whose free surface is observed with a commodity camera.
  arXiv  Detail & Related papers  (2022-03-11T07:01:23Z)
• Privacy-preserving machine learning with tensor networks [37.01494003138908]
  We show that tensor network architectures have especially prospective properties for privacy-preserving machine learning. First, we describe a new privacy vulnerability that is present in feedforward neural networks, illustrating it in synthetic and real-world datasets. We rigorously prove that such conditions are satisfied by tensor-network architectures.
  arXiv  Detail & Related papers  (2022-02-24T19:04:35Z)
• ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback [54.142719510638614]
  In this paper, we frame the problem of providing feedback as few-shot classification. A meta-learner adapts to give feedback to student code on a new programming question from just a few examples by instructors. Our approach was successfully deployed to deliver feedback to 16,000 student exam-solutions in a programming course offered by a tier 1 university.
  arXiv  Detail & Related papers  (2021-07-23T22:41:28Z)
• Jointly Modeling Heterogeneous Student Behaviors and Interactions Among Multiple Prediction Tasks [35.15654921278549]
  Prediction tasks about students have practical significance for both student and college. In this paper, we focus on modeling heterogeneous behaviors and making multiple predictions together. We design three motivating behavior prediction tasks based on a real-world dataset collected from a college.
  arXiv  Detail & Related papers  (2021-03-25T02:01:58Z)
• Explainable Empirical Risk Minimization [0.6299766708197883]
  Successful application of machine learning (ML) methods becomes increasingly dependent on their interpretability or explainability. This paper applies information-theoretic concepts to develop a novel measure for the subjective explainability of predictions delivered by a ML method. Our main contribution is the explainable empirical risk minimization (EERM) principle of learning a hypothesis that optimally balances between the subjective explainability and risk.
  arXiv  Detail & Related papers  (2020-09-03T07:16:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.