The Acquisition of Physical Knowledge in Generative Neural Networks

• URL: http://arxiv.org/abs/2310.19943v1
• Date: Mon, 30 Oct 2023 18:58:03 GMT
• Title: The Acquisition of Physical Knowledge in Generative Neural Networks
• Authors: Luca M. Schulze Buschoff, Eric Schulz, Marcel Binz
• Abstract summary: We investigate how the learning trajectories of deep generative neural networks compare to children's developmental trajectories using physical understanding as a testbed. We find that while our models are able to accurately predict a number of physical processes, their learning trajectories under both hypotheses do not follow the developmental trajectories of children.
• Score: 9.799950649945597
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As children grow older, they develop an intuitive understanding of the physical processes around them. Their physical understanding develops in stages, moving along developmental trajectories which have been mapped out extensively in previous empirical research. Here, we investigate how the learning trajectories of deep generative neural networks compare to children's developmental trajectories using physical understanding as a testbed. We outline an approach that allows us to examine two distinct hypotheses of human development - stochastic optimization and complexity increase. We find that while our models are able to accurately predict a number of physical processes, their learning trajectories under both hypotheses do not follow the developmental trajectories of children.

Related papers

• Contrastive Learning with Adaptive Neighborhoods for Brain Age Prediction on 3D Stiffness Maps [8.14243193774551]
  We introduce a novel contrastive loss that adapts dynamically during the training process, focusing on the localized neighborhoods of samples. This work presents the first application of self-supervised learning to brain mechanical properties, using compiled stiffness maps to predict brain age.
  arXiv  Detail & Related papers  (2024-08-01T12:58:19Z)
• Neural Lineage [56.34149480207817]
  We introduce a novel task known as neural lineage detection, aiming at discovering lineage relationships between parent and child models. For practical convenience, we introduce a learning-free approach, which integrates an approximation of the finetuning process into the neural network representation similarity metrics. For the pursuit of accuracy, we introduce a learning-based lineage detector comprising encoders and a transformer detector.
  arXiv  Detail & Related papers  (2024-06-17T01:11:53Z)
• Brain-Inspired Machine Intelligence: A Survey of Neurobiologically-Plausible Credit Assignment [65.268245109828]
  We examine algorithms for conducting credit assignment in artificial neural networks that are inspired or motivated by neurobiology. We organize the ever-growing set of brain-inspired learning schemes into six general families and consider these in the context of backpropagation of errors. The results of this review are meant to encourage future developments in neuro-mimetic systems and their constituent learning processes.
  arXiv  Detail & Related papers  (2023-12-01T05:20:57Z)
• Towards Self-Assembling Artificial Neural Networks through Neural Developmental Programs [10.524752369156339]
  Biological nervous systems are created in a fundamentally different way than current artificial neural networks. By contrast, biological nervous systems are grown through a dynamic self-organizing process.
  arXiv  Detail & Related papers  (2023-07-17T01:58:52Z)
• NeuroExplainer: Fine-Grained Attention Decoding to Uncover Cortical Development Patterns of Preterm Infants [73.85768093666582]
  We propose an explainable geometric deep network dubbed NeuroExplainer. NeuroExplainer is used to uncover altered infant cortical development patterns associated with preterm birth.
  arXiv  Detail & Related papers  (2023-01-01T12:48:12Z)
• Stochastic Gradient Descent Captures How Children Learn About Physics [0.0]
  We investigate how children's developmental trajectories compare to the learning trajectories of artificial systems. We find that the model's learning trajectory captures the developmental trajectories of children, thereby providing support to the idea of development as optimization.
  arXiv  Detail & Related papers  (2022-09-25T22:56:14Z)
• A Few-shot Learning Graph Multi-Trajectory Evolution Network for Forecasting Multimodal Baby Connectivity Development from a Baseline Timepoint [53.73316520733503]
  We propose a Graph Multi-Trajectory Evolution Network (GmTE-Net), which adopts a teacher-student paradigm. This is the first teacher-student architecture tailored for brain graph multi-trajectory growth prediction.
  arXiv  Detail & Related papers  (2021-10-06T08:26:57Z)
• Continual Developmental Neurosimulation Using Embodied Computational Agents [0.0]
  Continual developmental neurosimulation allows us to consider the role of developmental trajectories in bridging the related phenomena of nervous system morphogenesis, developmental learning, and plasticity. Our approach is tightly integrated with developmental embodiment, and can be implemented using a type of agent called developmental Braitenberg Vehicles (dBVs) dBVs begin their lives as a set of undefined structures that transform into agent-based systems including a body, sensors, effectors, and nervous system.
  arXiv  Detail & Related papers  (2021-03-07T07:22:49Z)
• A Developmental Neuro-Robotics Approach for Boosting the Recognition of Handwritten Digits [91.3755431537592]
  Recent evidence shows that a simulation of the children's embodied strategies can improve the machine intelligence too. This article explores the application of embodied strategies to convolutional neural network models in the context of developmental neuro-robotics.
  arXiv  Detail & Related papers  (2020-03-23T14:55:00Z)
• The large learning rate phase of deep learning: the catapult mechanism [50.23041928811575]
  We present a class of neural networks with solvable training dynamics. We find good agreement between our model's predictions and training dynamics in realistic deep learning settings. We believe our results shed light on characteristics of models trained at different learning rates.
  arXiv  Detail & Related papers  (2020-03-04T17:52:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.