AstroSpy: On detecting Fake Images in Astronomy via Joint Image-Spectral Representations

• URL: http://arxiv.org/abs/2407.06817v1
• Date: Tue, 9 Jul 2024 12:49:44 GMT
• Title: AstroSpy: On detecting Fake Images in Astronomy via Joint Image-Spectral Representations
• Authors: Mohammed Talha Alam, Raza Imam, Mohsen Guizani, Fakhri Karray,
• Abstract summary: The prevalence of AI-generated imagery has raised concerns about the authenticity of astronomical images. We present AstroSpy, a hybrid model that integrates both spectral and image features to distinguish real from synthetic astronomical images.
• Score: 31.75799061059914
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The prevalence of AI-generated imagery has raised concerns about the authenticity of astronomical images, especially with advanced text-to-image models like Stable Diffusion producing highly realistic synthetic samples. Existing detection methods, primarily based on convolutional neural networks (CNNs) or spectral analysis, have limitations when used independently. We present AstroSpy, a hybrid model that integrates both spectral and image features to distinguish real from synthetic astronomical images. Trained on a unique dataset of real NASA images and AI-generated fakes (approximately 18k samples), AstroSpy utilizes a dual-pathway architecture to fuse spatial and spectral information. This approach enables AstroSpy to achieve superior performance in identifying authentic astronomical images. Extensive evaluations demonstrate AstroSpy's effectiveness and robustness, significantly outperforming baseline models in both in-domain and cross-domain tasks, highlighting its potential to combat misinformation in astronomy.

Related papers

• Effective Fine-Tuning of Vision-Language Models for Accurate Galaxy Morphology Analysis [3.379005517804234]
  GalaxAlign is a novel method that fine-tunes pre-trained foundation models to achieve high accuracy on astronomical tasks. Our method extends a contrastive learning architecture to align three types of data in fine-tuning.
  arXiv  Detail & Related papers  (2024-11-29T05:10:47Z)
• Understanding and Improving Training-Free AI-Generated Image Detections with Vision Foundation Models [68.90917438865078]
  Deepfake techniques for facial synthesis and editing pose serious risks for generative models. In this paper, we investigate how detection performance varies across model backbones, types, and datasets. We introduce Contrastive Blur, which enhances performance on facial images, and MINDER, which addresses noise type bias, balancing performance across domains.
  arXiv  Detail & Related papers  (2024-11-28T13:04:45Z)
• Contrasting Deepfakes Diffusion via Contrastive Learning and Global-Local Similarities [88.398085358514]
  Contrastive Deepfake Embeddings (CoDE) is a novel embedding space specifically designed for deepfake detection. CoDE is trained via contrastive learning by additionally enforcing global-local similarities.
  arXiv  Detail & Related papers  (2024-07-29T18:00:10Z)
• Learned representation-guided diffusion models for large-image generation [58.192263311786824]
  We introduce a novel approach that trains diffusion models conditioned on embeddings from self-supervised learning (SSL) Our diffusion models successfully project these features back to high-quality histopathology and remote sensing images. Augmenting real data by generating variations of real images improves downstream accuracy for patch-level and larger, image-scale classification tasks.
  arXiv  Detail & Related papers  (2023-12-12T14:45:45Z)
• AstroCLIP: A Cross-Modal Foundation Model for Galaxies [40.43521617393482]
  AstroCLIP embeds galaxy images and spectra separately by pretraining separate transformer-based image and spectrum encoders in self-supervised settings. We find remarkable performance on all downstream tasks, even relative to supervised baselines. Our approach represents the first cross-modal self-supervised model for galaxies, and the first self-supervised transformer-based architectures for galaxy images and spectra.
  arXiv  Detail & Related papers  (2023-10-04T17:59:38Z)
• Combining multi-spectral data with statistical and deep-learning models for improved exoplanet detection in direct imaging at high contrast [39.90150176899222]
  Exoplanet signals can only be identified when combining several observations with dedicated detection algorithms. We learn a model of the spatial, temporal and spectral characteristics of the nuisance, directly from the observations. A convolutional neural network (CNN) is then trained in a supervised fashion to detect the residual signature of synthetic sources.
  arXiv  Detail & Related papers  (2023-06-21T13:42:07Z)
• Object Detection in Hyperspectral Image via Unified Spectral-Spatial Feature Aggregation [55.9217962930169]
  We present S2ADet, an object detector that harnesses the rich spectral and spatial complementary information inherent in hyperspectral images. S2ADet surpasses existing state-of-the-art methods, achieving robust and reliable results.
  arXiv  Detail & Related papers  (2023-06-14T09:01:50Z)
• GM-NeRF: Learning Generalizable Model-based Neural Radiance Fields from Multi-view Images [79.39247661907397]
  We introduce an effective framework Generalizable Model-based Neural Radiance Fields to synthesize free-viewpoint images. Specifically, we propose a geometry-guided attention mechanism to register the appearance code from multi-view 2D images to a geometry proxy.
  arXiv  Detail & Related papers  (2023-03-24T03:32:02Z)
• Realistic galaxy image simulation via score-based generative models [0.0]
  We show that a score-based generative model can be used to produce realistic yet fake images that mimic observations of galaxies. Subjectively, the generated galaxies are highly realistic when compared with samples from the real dataset.
  arXiv  Detail & Related papers  (2021-11-02T16:27:08Z)
• Low-Rank Subspaces in GANs [101.48350547067628]
  This work introduces low-rank subspaces that enable more precise control of GAN generation. LowRankGAN is able to find the low-dimensional representation of attribute manifold. Experiments on state-of-the-art GAN models (including StyleGAN2 and BigGAN) trained on various datasets demonstrate the effectiveness of our LowRankGAN.
  arXiv  Detail & Related papers  (2021-06-08T16:16:32Z)
• Hyperspectral and multispectral image fusion under spectrally varying spatial blurs -- Application to high dimensional infrared astronomical imaging [11.243400478302767]
  We propose a data fusion method which combines the benefits of each image to recover a high-spectral resolution data variant. We conduct experiments on a realistic synthetic dataset of simulated observation of the upcoming James Webb Space Telescope.
  arXiv  Detail & Related papers  (2019-12-26T13:58:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.