Exploring the Limits of Fine-grained LLM-based Physics Inference via Premise Removal Interventions

• URL: http://arxiv.org/abs/2404.18384v2
• Date: Tue, 01 Oct 2024 06:17:52 GMT
• Title: Exploring the Limits of Fine-grained LLM-based Physics Inference via Premise Removal Interventions
• Authors: Jordan Meadows, Tamsin James, Andre Freitas,
• Abstract summary: Language models (LMs) can hallucinate when performing complex mathematical reasoning. Physical context requires that any symbolic manipulation satisfies complex semantics. We show that LMs' mathematical reasoning is not physics-informed in this setting.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Language models (LMs) can hallucinate when performing complex mathematical reasoning. Physics provides a rich domain for assessing their mathematical capabilities, where physical context requires that any symbolic manipulation satisfies complex semantics (\textit{e.g.,} units, tensorial order). In this work, we systematically remove crucial context from prompts to force instances where model inference may be algebraically coherent, yet unphysical. We assess LM capabilities in this domain using a curated dataset encompassing multiple notations and Physics subdomains. Further, we improve zero-shot scores using synthetic in-context examples, and demonstrate non-linear degradation of derivation quality with perturbation strength via the progressive omission of supporting premises. We find that the models' mathematical reasoning is not physics-informed in this setting, where physical context is predominantly ignored in favour of reverse-engineering solutions.

Related papers

• Partition of Unity Physics-Informed Neural Networks (POU-PINNs): An Unsupervised Framework for Physics-Informed Domain Decomposition and Mixtures of Experts [6.179530974508392]
  This study presents a novel unsupervised learning framework that identifies spatial inverse with specific governing physics. A vital feature of this method is a physics residual-based loss function that detects variations in physical properties without requiring labeled data. Its effectiveness is demonstrated through applications in porous thermal media ablation and ice-sheet modeling.
  arXiv  Detail & Related papers  (2024-12-07T16:07:43Z)
• LLMPhy: Complex Physical Reasoning Using Large Language Models and World Models [35.01842161084472]
  We propose a new physical reasoning task and a dataset, dubbed TraySim. Our task involves predicting the dynamics of several objects on a tray that is given an external impact. We present LLMPhy, a zero-shot black-box optimization framework that leverages the physics knowledge and program synthesis abilities of LLMs. Our results show that the combination of the LLM and the physics engine leads to state-of-the-art zero-shot physical reasoning performance.
  arXiv  Detail & Related papers  (2024-11-12T18:56:58Z)
• DimOL: Dimensional Awareness as A New 'Dimension' in Operator Learning [63.5925701087252]
  We introduce DimOL (Dimension-aware Operator Learning), drawing insights from dimensional analysis. To implement DimOL, we propose the ProdLayer, which can be seamlessly integrated into FNO-based and Transformer-based PDE solvers. Empirically, DimOL models achieve up to 48% performance gain within the PDE datasets.
  arXiv  Detail & Related papers  (2024-10-08T10:48:50Z)
• Physics-Informed Graph-Mesh Networks for PDEs: A hybrid approach for complex problems [0.24578723416255746]
  We introduce a hybrid approach combining physics-informed graph neural networks with numerical kernels from finite elements. After studying the theoretical properties of our model, we apply it to complex geometries, in two and three dimensions. Our choices are supported by an ablation study, and we evaluate the generalisation capacity of the proposed approach.
  arXiv  Detail & Related papers  (2024-09-25T07:52:29Z)
• On the Dynamics Under the Unhinged Loss and Beyond [104.49565602940699]
  We introduce the unhinged loss, a concise loss function, that offers more mathematical opportunities to analyze closed-form dynamics. The unhinged loss allows for considering more practical techniques, such as time-vary learning rates and feature normalization.
  arXiv  Detail & Related papers  (2023-12-13T02:11:07Z)
• Discovering Interpretable Physical Models using Symbolic Regression and Discrete Exterior Calculus [55.2480439325792]
  We propose a framework that combines Symbolic Regression (SR) and Discrete Exterior Calculus (DEC) for the automated discovery of physical models. DEC provides building blocks for the discrete analogue of field theories, which are beyond the state-of-the-art applications of SR to physical problems. We prove the effectiveness of our methodology by re-discovering three models of Continuum Physics from synthetic experimental data.
  arXiv  Detail & Related papers  (2023-10-10T13:23:05Z)
• Deep symbolic regression for physics guided by units constraints: toward the automated discovery of physical laws [0.0]
  Symbolic Regression is the study of algorithms that automate the search for analytic expressions that fit data. We present $Phi$-SO, a framework for recovering analytical symbolic expressions from physics data.
  arXiv  Detail & Related papers  (2023-03-06T16:47:59Z)
• Generalized Neural Closure Models with Interpretability [28.269731698116257]
  We develop a novel and versatile methodology of unified neural partial delay differential equations. We augment existing/low-fidelity dynamical models directly in their partial differential equation (PDE) forms with both Markovian and non-Markovian neural network (NN) closure parameterizations. We demonstrate the new generalized neural closure models (gnCMs) framework using four sets of experiments based on advecting nonlinear waves, shocks, and ocean acidification models.
  arXiv  Detail & Related papers  (2023-01-15T21:57:43Z)
• Symmetry Group Equivariant Architectures for Physics [52.784926970374556]
  In the domain of machine learning, an awareness of symmetries has driven impressive performance breakthroughs. We argue that both the physics community and the broader machine learning community have much to understand.
  arXiv  Detail & Related papers  (2022-03-11T18:27:04Z)
• Physics informed neural networks for continuum micromechanics [68.8204255655161]
  Recently, physics informed neural networks have successfully been applied to a broad variety of problems in applied mathematics and engineering. Due to the global approximation, physics informed neural networks have difficulties in displaying localized effects and strong non-linear solutions by optimization. It is shown, that the domain decomposition approach is able to accurately resolve nonlinear stress, displacement and energy fields in heterogeneous microstructures obtained from real-world $mu$CT-scans.
  arXiv  Detail & Related papers  (2021-10-14T14:05:19Z)
• Scalable Differentiable Physics for Learning and Control [99.4302215142673]
  Differentiable physics is a powerful approach to learning and control problems that involve physical objects and environments. We develop a scalable framework for differentiable physics that can support a large number of objects and their interactions.
  arXiv  Detail & Related papers  (2020-07-04T19:07:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.