Learnable wavelet neural networks for cosmological inference
- URL: http://arxiv.org/abs/2307.14362v1
- Date: Mon, 24 Jul 2023 22:12:16 GMT
- Title: Learnable wavelet neural networks for cosmological inference
- Authors: Christian Pedersen, Michael Eickenberg, Shirley Ho
- Abstract summary: Learnable scattering transform is a kind of convolutional neural network that uses trainable wavelets as filters.
We present two models based on the scattering transform, one constructed for performance, and one constructed for interpretability.
We find that scattering architectures are able to outperform a CNN, significantly in the case of small training data samples.
- Score: 5.1268138545584145
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Convolutional neural networks (CNNs) have been shown to both extract more
information than the traditional two-point statistics from cosmological fields,
and marginalise over astrophysical effects extremely well. However, CNNs
require large amounts of training data, which is potentially problematic in the
domain of expensive cosmological simulations, and it is difficult to interpret
the network. In this work we apply the learnable scattering transform, a kind
of convolutional neural network that uses trainable wavelets as filters, to the
problem of cosmological inference and marginalisation over astrophysical
effects. We present two models based on the scattering transform, one
constructed for performance, and one constructed for interpretability, and
perform a comparison with a CNN. We find that scattering architectures are able
to outperform a CNN, significantly in the case of small training data samples.
Additionally we present a lightweight scattering network that is highly
interpretable.
Related papers
- CNN2GNN: How to Bridge CNN with GNN [59.42117676779735]
We propose a novel CNN2GNN framework to unify CNN and GNN together via distillation.
The performance of distilled boosted'' two-layer GNN on Mini-ImageNet is much higher than CNN containing dozens of layers such as ResNet152.
arXiv Detail & Related papers (2024-04-23T08:19:08Z) - Transferability of Convolutional Neural Networks in Stationary Learning
Tasks [96.00428692404354]
We introduce a novel framework for efficient training of convolutional neural networks (CNNs) for large-scale spatial problems.
We show that a CNN trained on small windows of such signals achieves a nearly performance on much larger windows without retraining.
Our results show that the CNN is able to tackle problems with many hundreds of agents after being trained with fewer than ten.
arXiv Detail & Related papers (2023-07-21T13:51:45Z) - Solving Large-scale Spatial Problems with Convolutional Neural Networks [88.31876586547848]
We employ transfer learning to improve training efficiency for large-scale spatial problems.
We propose that a convolutional neural network (CNN) can be trained on small windows of signals, but evaluated on arbitrarily large signals with little to no performance degradation.
arXiv Detail & Related papers (2023-06-14T01:24:42Z) - Using explainability to design physics-aware CNNs for solving subsurface
inverse problems [0.0]
We present a novel method of using explainability techniques to design physics-aware neural networks.
We demonstrate our approach by developing a convolutional neural network (CNN) for solving an inverse problem for shallow subsurface imaging.
arXiv Detail & Related papers (2022-11-16T03:59:29Z) - Interpreting convolutional neural networks' low dimensional
approximation to quantum spin systems [1.631115063641726]
Convolutional neural networks (CNNs) have been employed along with Variational Monte Carlo methods for finding the ground state of quantum many-body spin systems.
We provide a theoretical and experimental analysis of how the CNN optimize learning for spin systems, and investigate the CNN's low dimensional approximation.
Our results allow us to gain a comprehensive, improved understanding of how CNNs successfully approximate quantum spin Hamiltonians.
arXiv Detail & Related papers (2022-10-03T02:49:16Z) - Physics-informed neural networks for diffraction tomography [0.1199955563466263]
We propose a physics-informed neural network as the forward model for tomographic reconstructions of biological samples.
By training this network with the Helmholtz equation as a physical loss, we can predict the scattered field accurately.
arXiv Detail & Related papers (2022-07-28T16:56:50Z) - Deep Architecture Connectivity Matters for Its Convergence: A
Fine-Grained Analysis [94.64007376939735]
We theoretically characterize the impact of connectivity patterns on the convergence of deep neural networks (DNNs) under gradient descent training.
We show that by a simple filtration on "unpromising" connectivity patterns, we can trim down the number of models to evaluate.
arXiv Detail & Related papers (2022-05-11T17:43:54Z) - Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs.
By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
arXiv Detail & Related papers (2022-02-01T17:11:13Z) - How to quantify fields or textures? A guide to the scattering transform [0.0]
We advocate for the use of the scattering transform (Mallat 2012), a powerful statistic which borrows mathematical ideas from CNNs but does not require any training, and is interpretable.
We show that it provides a relatively compact set of summary statistics with visual interpretation and which carries most of the relevant information in a wide range of scientific applications.
arXiv Detail & Related papers (2021-11-30T22:11:54Z) - Estimating permeability of 3D micro-CT images by physics-informed CNNs
based on DNS [1.6274397329511197]
This paper presents a novel methodology for permeability prediction from micro-CT scans of geological rock samples.
The training data set for CNNs dedicated to permeability prediction consists of permeability labels that are typically generated by classical lattice Boltzmann methods (LBM)
We instead perform direct numerical simulation (DNS) by solving the stationary Stokes equation in an efficient and distributed-parallel manner.
arXiv Detail & Related papers (2021-09-04T08:43:19Z) - Network Diffusions via Neural Mean-Field Dynamics [52.091487866968286]
We propose a novel learning framework for inference and estimation problems of diffusion on networks.
Our framework is derived from the Mori-Zwanzig formalism to obtain an exact evolution of the node infection probabilities.
Our approach is versatile and robust to variations of the underlying diffusion network models.
arXiv Detail & Related papers (2020-06-16T18:45:20Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.