Augmented Physics: Creating Interactive and Embedded Physics Simulations from Static Textbook Diagrams

• URL: http://arxiv.org/abs/2405.18614v2
• Date: Sat, 10 Aug 2024 16:37:38 GMT
• Title: Augmented Physics: Creating Interactive and Embedded Physics Simulations from Static Textbook Diagrams
• Authors: Aditya Gunturu, Yi Wen, Nandi Zhang, Jarin Thundathil, Rubaiat Habib Kazi, Ryo Suzuki,
• Abstract summary: We introduce Augmented Physics, a machine learning-integrated authoring tool designed for creating embedded interactive physics simulations from static textbook diagrams. Our system enables users to semi-automatically extract diagrams from physics textbooks and generate interactive simulations based on the extracted content.
• Score: 10.883862721068223
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce Augmented Physics, a machine learning-integrated authoring tool designed for creating embedded interactive physics simulations from static textbook diagrams. Leveraging recent advancements in computer vision, such as Segment Anything and Multi-modal LLMs, our web-based system enables users to semi-automatically extract diagrams from physics textbooks and generate interactive simulations based on the extracted content. These interactive diagrams are seamlessly integrated into scanned textbook pages, facilitating interactive and personalized learning experiences across various physics concepts, such as optics, circuits, and kinematics. Drawing from an elicitation study with seven physics instructors, we explore four key augmentation strategies: 1) augmented experiments, 2) animated diagrams, 3) bi-directional binding, and 4) parameter visualization. We evaluate our system through technical evaluation, a usability study (N=12), and expert interviews (N=12). Study findings suggest that our system can facilitate more engaging and personalized learning experiences in physics education.

Related papers

• Latent Intuitive Physics: Learning to Transfer Hidden Physics from A 3D Video [58.043569985784806]
  We introduce latent intuitive physics, a transfer learning framework for physics simulation. It can infer hidden properties of fluids from a single 3D video and simulate the observed fluid in novel scenes. We validate our model in three ways: (i) novel scene simulation with the learned visual-world physics, (ii) future prediction of the observed fluid dynamics, and (iii) supervised particle simulation.
  arXiv  Detail & Related papers  (2024-06-18T16:37:44Z)
• A Personalised Learning Tool for Physics Undergraduate Students Built On a Large Language Model for Symbolic Regression [0.6666419797034796]
  Interleaved practice enhances the memory and problem-solving ability of students in undergraduate courses. We introduce a personalized learning tool built on a Large Language Model (LLM) that can provide immediate and personalized attention to students.
  arXiv  Detail & Related papers  (2024-06-17T13:43:30Z)
• Physics-Encoded Graph Neural Networks for Deformation Prediction under Contact [87.69278096528156]
  In robotics, it's crucial to understand object deformation during tactile interactions. We introduce a method using Physics-Encoded Graph Neural Networks (GNNs) for such predictions. We've made our code and dataset public to advance research in robotic simulation and grasping.
  arXiv  Detail & Related papers  (2024-02-05T19:21:52Z)
• Augmented Math: Authoring AR-Based Explorable Explanations by Augmenting Static Math Textbooks [1.8097223019080158]
  We introduce Augmented Math, a machine learning-based approach to authoring AR explorable explanations by augmenting static math textbooks without programming. To augment a static document, our system first extracts mathematical formulas and figures from a given document using optical character recognition (OCR) and computer vision. This empowers non-technical users, such as teachers or students, to transform existing math textbooks and handouts into on-demand and personalized explorable explanations.
  arXiv  Detail & Related papers  (2023-07-30T03:02:52Z)
• Multimodal Lecture Presentations Dataset: Understanding Multimodality in Educational Slides [57.86931911522967]
  We test the capabilities of machine learning models in multimodal understanding of educational content. Our dataset contains aligned slides and spoken language, for 180+ hours of video and 9000+ slides, with 10 lecturers from various subjects. We introduce PolyViLT, a multimodal transformer trained with a multi-instance learning loss that is more effective than current approaches.
  arXiv  Detail & Related papers  (2022-08-17T05:30:18Z)
• Dynamic Visual Reasoning by Learning Differentiable Physics Models from Video and Language [92.7638697243969]
  We propose a unified framework that can jointly learn visual concepts and infer physics models of objects from videos and language. This is achieved by seamlessly integrating three components: a visual perception module, a concept learner, and a differentiable physics engine.
  arXiv  Detail & Related papers  (2021-10-28T17:59:13Z)
• ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation [75.0278287071591]
  ThreeDWorld (TDW) is a platform for interactive multi-modal physical simulation. TDW enables simulation of high-fidelity sensory data and physical interactions between mobile agents and objects in rich 3D environments. We present initial experiments enabled by TDW in emerging research directions in computer vision, machine learning, and cognitive science.
  arXiv  Detail & Related papers  (2020-07-09T17:33:27Z)
• Quantum Composer: A programmable quantum visualization and simulation tool for education and research [0.0]
  Quantum Composer allows the user to build, expand, or explore quantum mechanical simulations by interacting with graphically connectable nodes. We illustrate its open-ended applicability in both introductory and advanced quantum mechanics courses, student projects, and for visual exploration within research environments.
  arXiv  Detail & Related papers  (2020-06-12T15:19:02Z)
• Occlusion resistant learning of intuitive physics from videos [52.25308231683798]
  Key ability for artificial systems is to understand physical interactions between objects, and predict future outcomes of a situation. This ability, often referred to as intuitive physics, has recently received attention and several methods were proposed to learn these physical rules from video sequences.
  arXiv  Detail & Related papers  (2020-04-30T19:35:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.