DeepAstroUDA: Semi-Supervised Universal Domain Adaptation for Cross-Survey Galaxy Morphology Classification and Anomaly Detection

• URL: http://arxiv.org/abs/2302.02005v2
• Date: Wed, 22 Mar 2023 17:03:51 GMT
• Title: DeepAstroUDA: Semi-Supervised Universal Domain Adaptation for Cross-Survey Galaxy Morphology Classification and Anomaly Detection
• Authors: A. \'Ciprijanovi\'c, A. Lewis, K. Pedro, S. Madireddy, B. Nord, G. N. Perdue, S. M. Wild
• Abstract summary: We present a universal domain adaptation method, textitDeepAstroUDA, as an approach to overcome this challenge. textitDeepAstroUDA is capable of bridging the gap between two astronomical surveys, increasing classification accuracy in both domains. Our method also performs well as an anomaly detection algorithm and successfully clusters unknown class samples even in the unlabeled target dataset.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Artificial intelligence methods show great promise in increasing the quality and speed of work with large astronomical datasets, but the high complexity of these methods leads to the extraction of dataset-specific, non-robust features. Therefore, such methods do not generalize well across multiple datasets. We present a universal domain adaptation method, \textit{DeepAstroUDA}, as an approach to overcome this challenge. This algorithm performs semi-supervised domain adaptation and can be applied to datasets with different data distributions and class overlaps. Non-overlapping classes can be present in any of the two datasets (the labeled source domain, or the unlabeled target domain), and the method can even be used in the presence of unknown classes. We apply our method to three examples of galaxy morphology classification tasks of different complexities ($3$-class and $10$-class problems), with anomaly detection: 1) datasets created after different numbers of observing years from a single survey (LSST mock data of $1$ and $10$ years of observations); 2) data from different surveys (SDSS and DECaLS); and 3) data from observing fields with different depths within one survey (wide field and Stripe 82 deep field of SDSS). For the first time, we demonstrate the successful use of domain adaptation between very discrepant observational datasets. \textit{DeepAstroUDA} is capable of bridging the gap between two astronomical surveys, increasing classification accuracy in both domains (up to $40\%$ on the unlabeled data), and making model performance consistent across datasets. Furthermore, our method also performs well as an anomaly detection algorithm and successfully clusters unknown class samples even in the unlabeled target dataset.

Related papers

• Uni$^2$Det: Unified and Universal Framework for Prompt-Guided Multi-dataset 3D Detection [64.08296187555095]
  Uni$2$Det is a framework for unified and universal multi-dataset training on 3D detection. We introduce multi-stage prompting modules for multi-dataset 3D detection. Results on zero-shot cross-dataset transfer validate the generalization capability of our proposed method.
  arXiv  Detail & Related papers  (2024-09-30T17:57:50Z)
• A Novel Adaptive Fine-Tuning Algorithm for Multimodal Models: Self-Optimizing Classification and Selection of High-Quality Datasets in Remote Sensing [46.603157010223505]
  We propose an adaptive fine-tuning algorithm for multimodal large models. We train the model on two 3090 GPU using one-third of the GeoChat multimodal remote sensing dataset. The model achieved scores of 89.86 and 77.19 on the UCMerced and AID evaluation datasets.
  arXiv  Detail & Related papers  (2024-09-20T09:19:46Z)
• Anno-incomplete Multi-dataset Detection [67.69438032767613]
  We propose a novel problem as "-incomplete Multi-dataset Detection" We develop an end-to-end multi-task learning architecture which can accurately detect all the object categories with multiple partially annotated datasets.
  arXiv  Detail & Related papers  (2024-08-29T03:58:21Z)
• ARC: A Generalist Graph Anomaly Detector with In-Context Learning [62.202323209244]
  ARC is a generalist GAD approach that enables a one-for-all'' GAD model to detect anomalies across various graph datasets on-the-fly. equipped with in-context learning, ARC can directly extract dataset-specific patterns from the target dataset. Extensive experiments on multiple benchmark datasets from various domains demonstrate the superior anomaly detection performance, efficiency, and generalizability of ARC.
  arXiv  Detail & Related papers  (2024-05-27T02:42:33Z)
• Generalized Category Discovery with Clustering Assignment Consistency [56.92546133591019]
  Generalized category discovery (GCD) is a recently proposed open-world task. We propose a co-training-based framework that encourages clustering consistency. Our method achieves state-of-the-art performance on three generic benchmarks and three fine-grained visual recognition datasets.
  arXiv  Detail & Related papers  (2023-10-30T00:32:47Z)
• SeMAnD: Self-Supervised Anomaly Detection in Multimodal Geospatial Datasets [0.0]
  SeMAnD is able to detect real-world defects and outperforms domain-agnostic anomaly detection strategies by 4.8-19.7%. We show that model performance increases (i) up to 20.4% as the number of input modalities increase and (ii) up to 22.9% as the diversity and strength of training data augmentations increase.
  arXiv  Detail & Related papers  (2023-09-26T20:18:31Z)
• Semi-Supervised Domain Adaptation for Cross-Survey Galaxy Morphology Classification and Anomaly Detection [57.85347204640585]
  We develop a Universal Domain Adaptation method DeepAstroUDA. It can be applied to datasets with different types of class overlap. For the first time, we demonstrate the successful use of domain adaptation on two very different observational datasets.
  arXiv  Detail & Related papers  (2022-11-01T18:07:21Z)
• Tackling Long-Tailed Category Distribution Under Domain Shifts [50.21255304847395]
  Existing approaches cannot handle the scenario where both issues exist. We designed three novel core functional blocks including Distribution Calibrated Classification Loss, Visual-Semantic Mapping and Semantic-Similarity Guided Augmentation. Two new datasets were proposed for this problem, named AWA2-LTS and ImageNet-LTS.
  arXiv  Detail & Related papers  (2022-07-20T19:07:46Z)
• A Topological Approach for Semi-Supervised Learning [0.0]
  We present new semi-supervised learning methods based on techniques from Topological Data Analysis (TDA) In particular, we have created two semi-supervised learning methods following two different topological approaches. The results show that the methods developed in this work outperform both the results obtained with models trained with only manually labelled data, and those obtained with classical semi-supervised learning methods.
  arXiv  Detail & Related papers  (2022-05-19T15:23:39Z)
• BikNN: Anomaly Estimation in Bilateral Domains with k-Nearest Neighbors [1.2183405753834562]
  A novel framework for anomaly estimation is proposed in this paper. We attempt to estimate the degree of anomaly in both spatial and density domains. Our method takes into account both the spatial domain and the density domain and can be adapted to different datasets by adjusting a few parameters manually.
  arXiv  Detail & Related papers  (2021-05-11T13:45:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.