Finding Strong Gravitational Lenses Through Self-Attention

• URL: http://arxiv.org/abs/2110.09202v1
• Date: Mon, 18 Oct 2021 11:40:48 GMT
• Title: Finding Strong Gravitational Lenses Through Self-Attention
• Abstract summary: We propose a new automated architecture based on the principle of self-attention to find strong gravitational lensing. We constructed 21 self-attention based encoder models and four convolution neural networks trained to identify gravitational lenses from the Bologna Lens Challenge.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The upcoming large scale surveys are expected to find approximately $10^5$ strong gravitational systems by analyzing data of many orders of magnitude than the current era. In this scenario, non-automated techniques will be highly challenging and time-consuming. We propose a new automated architecture based on the principle of self-attention to find strong gravitational lensing. The advantages of self-attention based encoder models over convolution neural networks are investigated and encoder models are analyzed to optimize performance. We constructed 21 self-attention based encoder models and four convolution neural networks trained to identify gravitational lenses from the Bologna Lens Challenge. Each model is trained separately using 18,000 simulated images, cross-validated using 2 000 images, and then applied to a test set with 100 000 images. We used four different metrics for evaluation: classification accuracy, the area under the receiver operating characteristic curve (AUROC), the $TPR_0$ score and the $TPR_{10}$ score. The performance of the self-attention based encoder models and CNN's participated in the challenge are compared. The encoder models performed better than the CNNs and surpassed the CNN models that participated in the bologna lens challenge by a high margin for the $TPR_0$ and $TPR_{10}$. In terms of the AUROC, the encoder models scored equivalent to the top CNN model by only using one-sixth parameters to that of the CNN. Self-Attention based models have a clear advantage compared to simpler CNNs. A low computational cost and complexity make it a highly competing architecture to currently used residual neural networks. Moreover, introducing the encoder layers can also tackle the over-fitting problem present in the CNN's by acting as effective filters.

Related papers

• Graph Neural Network for Accurate and Low-complexity SAR ATR [2.9766397696234996]
  We propose a graph neural network (GNN) model to achieve accurate and low-latency SAR ATR. The proposed GNN model has low computation complexity and achieves comparable high accuracy. Compared with the state-of-the-art CNNs, the proposed GNN model has only 1/3000 computation cost and 1/80 model size.
  arXiv  Detail & Related papers  (2023-05-11T20:17:41Z)
• InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions [95.94629864981091]
  This work presents a new large-scale CNN-based foundation model, termed InternImage, which can obtain the gain from increasing parameters and training data like ViTs. The proposed InternImage reduces the strict inductive bias of traditional CNNs and makes it possible to learn stronger and more robust patterns with large-scale parameters from massive data like ViTs.
  arXiv  Detail & Related papers  (2022-11-10T18:59:04Z)
• Verifiable and Energy Efficient Medical Image Analysis with Quantised Self-attentive Deep Neural Networks [7.751856268560217]
  We propose quantised stand-alone self-attention based models as an alternative to traditional CNNs. We experimentally validate the performance of our method on classification and segmentation tasks.
  arXiv  Detail & Related papers  (2022-09-30T07:51:29Z)
• Deep Learning based Coverage and Rate Manifold Estimation in Cellular Networks [34.83409329913271]
  CNN-AE is trained using location data of India, Brazil, Germany, and the USA. CNN-AE improves the coverage and rate prediction errors by a margin of as large as $40%$ and $25%$ respectively.
  arXiv  Detail & Related papers  (2022-02-13T19:26:44Z)
• Exploiting Hybrid Models of Tensor-Train Networks for Spoken Command Recognition [9.262289183808035]
  This work aims to design a low complexity spoken command recognition (SCR) system. We exploit a deep hybrid architecture of a tensor-train (TT) network to build an end-to-end SRC pipeline. Our proposed CNN+(TT-DNN) model attains a competitive accuracy of 96.31% with 4 times fewer model parameters than the CNN model.
  arXiv  Detail & Related papers  (2022-01-11T05:57:38Z)
• BreakingBED -- Breaking Binary and Efficient Deep Neural Networks by Adversarial Attacks [65.2021953284622]
  We study robustness of CNNs against white-box and black-box adversarial attacks. Results are shown for distilled CNNs, agent-based state-of-the-art pruned models, and binarized neural networks.
  arXiv  Detail & Related papers  (2021-03-14T20:43:19Z)
• The Mind's Eye: Visualizing Class-Agnostic Features of CNNs [92.39082696657874]
  We propose an approach to visually interpret CNN features given a set of images by creating corresponding images that depict the most informative features of a specific layer. Our method uses a dual-objective activation and distance loss, without requiring a generator network nor modifications to the original model.
  arXiv  Detail & Related papers  (2021-01-29T07:46:39Z)
• On the Performance of Convolutional Neural Networks under High and Low Frequency Information [13.778851745408133]
  We study the performance of CNN models over the high and low frequency information of the images. We propose the filtering based data augmentation during training. A satisfactory performance improvement has been observed in terms of robustness and low frequency generalization.
  arXiv  Detail & Related papers  (2020-10-30T17:54:45Z)
• Deep learning for gravitational-wave data analysis: A resampling white-box approach [62.997667081978825]
  We apply Convolutional Neural Networks (CNNs) to detect gravitational wave (GW) signals of compact binary coalescences, using single-interferometer data from LIGO detectors. CNNs were quite precise to detect noise but not sensitive enough to recall GW signals, meaning that CNNs are better for noise reduction than generation of GW triggers.
  arXiv  Detail & Related papers  (2020-09-09T03:28:57Z)
• Exploring Deep Hybrid Tensor-to-Vector Network Architectures for Regression Based Speech Enhancement [53.47564132861866]
  We find that a hybrid architecture, namely CNN-TT, is capable of maintaining a good quality performance with a reduced model parameter size. CNN-TT is composed of several convolutional layers at the bottom for feature extraction to improve speech quality.
  arXiv  Detail & Related papers  (2020-07-25T22:21:05Z)
• Approximation and Non-parametric Estimation of ResNet-type Convolutional Neural Networks [52.972605601174955]
  We show a ResNet-type CNN can attain the minimax optimal error rates in important function classes. We derive approximation and estimation error rates of the aformentioned type of CNNs for the Barron and H"older classes.
  arXiv  Detail & Related papers  (2019-03-24T19:42:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.