Distinguishing Natural and Computer-Generated Images using Multi-Colorspace fused EfficientNet

• URL: http://arxiv.org/abs/2110.09428v1
• Date: Mon, 18 Oct 2021 15:55:45 GMT
• Title: Distinguishing Natural and Computer-Generated Images using Multi-Colorspace fused EfficientNet
• Authors: Manjary P Gangan, Anoop K, and Lajish V L
• Abstract summary: In a real-world image forensic scenario, it is highly essential to consider all categories of image generation. We propose a Multi-Colorspace fused EfficientNet model by parallelly fusing three EfficientNet networks. Our model outperforms the baselines in terms of accuracy, robustness towards post-processing, and generalizability towards other datasets.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The problem of distinguishing natural images from photo-realistic computer-generated ones either addresses natural images versus computer graphics or natural images versus GAN images, at a time. But in a real-world image forensic scenario, it is highly essential to consider all categories of image generation, since in most cases image generation is unknown. We, for the first time, to our best knowledge, approach the problem of distinguishing natural images from photo-realistic computer-generated images as a three-class classification task classifying natural, computer graphics, and GAN images. For the task, we propose a Multi-Colorspace fused EfficientNet model by parallelly fusing three EfficientNet networks that follow transfer learning methodology where each network operates in different colorspaces, RGB, LCH, and HSV, chosen after analyzing the efficacy of various colorspace transformations in this image forensics problem. Our model outperforms the baselines in terms of accuracy, robustness towards post-processing, and generalizability towards other datasets. We conduct psychophysics experiments to understand how accurately humans can distinguish natural, computer graphics, and GAN images where we could observe that humans find difficulty in classifying these images, particularly the computer-generated images, indicating the necessity of computational algorithms for the task. We also analyze the behavior of our model through visual explanations to understand salient regions that contribute to the model's decision making and compare with manual explanations provided by human participants in the form of region markings, where we could observe similarities in both the explanations indicating the powerful nature of our model to take the decisions meaningfully.

Related papers

• Novel computational workflows for natural and biomedical image processing based on hypercomplex algebras [49.81327385913137]
  Hypercomplex image processing extends conventional techniques in a unified paradigm encompassing algebraic and geometric principles. This workleverages quaternions and the two-dimensional planes split framework (splitting of a quaternion - representing a pixel - into pairs of 2D planes) for natural/biomedical image analysis. The proposed can regulate color appearance (e.g. with alternative renditions and grayscale conversion) and image contrast, be part of automated image processing pipelines.
  arXiv  Detail & Related papers  (2025-02-11T18:38:02Z)
• MatchAnything: Universal Cross-Modality Image Matching with Large-Scale Pre-Training [62.843316348659165]
  Deep learning-based image matching algorithms have dramatically outperformed humans in rapidly and accurately finding large amounts of correspondences. We propose a large-scale pre-training framework that utilizes synthetic cross-modal training signals to train models to recognize and match fundamental structures across images. Our key finding is that the matching model trained with our framework achieves remarkable generalizability across more than eight unseen cross-modality registration tasks.
  arXiv  Detail & Related papers  (2025-01-13T18:37:36Z)
• Convolution goes higher-order: a biologically inspired mechanism empowers image classification [0.8999666725996975]
  We propose a novel approach to image classification inspired by complex nonlinear biological visual processing. Our model incorporates a Volterra-like expansion of the convolution operator, capturing multiplicative interactions. Our work bridges neuroscience and deep learning, offering a path towards more effective, biologically inspired computer vision models.
  arXiv  Detail & Related papers  (2024-12-09T18:33:09Z)
• A Robust Image Forensic Framework Utilizing Multi-Colorspace Enriched Vision Transformer for Distinguishing Natural and Computer-Generated Images [0.0]
  We propose a robust forensic classifier framework leveraging enriched vision transformers to distinguish between natural and generated images. Our approach outperforms baselines, demonstrating 94.25% test accuracy with significant performance gains in individual class accuracies. This work advances the state-of-the-art in image forensics by providing a generalized and resilient solution to distinguish between natural and generated images.
  arXiv  Detail & Related papers  (2023-08-14T17:11:17Z)
• Multi-Domain Norm-referenced Encoding Enables Data Efficient Transfer Learning of Facial Expression Recognition [62.997667081978825]
  We propose a biologically-inspired mechanism for transfer learning in facial expression recognition. Our proposed architecture provides an explanation for how the human brain might innately recognize facial expressions on varying head shapes. Our model achieves a classification accuracy of 92.15% on the FERG dataset with extreme data efficiency.
  arXiv  Detail & Related papers  (2023-04-05T09:06:30Z)
• A domain adaptive deep learning solution for scanpath prediction of paintings [66.46953851227454]
  This paper focuses on the eye-movement analysis of viewers during the visual experience of a certain number of paintings. We introduce a new approach to predicting human visual attention, which impacts several cognitive functions for humans. The proposed new architecture ingests images and returns scanpaths, a sequence of points featuring a high likelihood of catching viewers' attention.
  arXiv  Detail & Related papers  (2022-09-22T22:27:08Z)
• Bridging Composite and Real: Towards End-to-end Deep Image Matting [88.79857806542006]
  We study the roles of semantics and details for image matting. We propose a novel Glance and Focus Matting network (GFM), which employs a shared encoder and two separate decoders. Comprehensive empirical studies have demonstrated that GFM outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-10-30T10:57:13Z)
• Cross-View Image Synthesis with Deformable Convolution and Attention Mechanism [29.528402825356398]
  We propose to use Generative Adversarial Networks(GANs) based on a deformable convolution and attention mechanism to solve the problem of cross-view image synthesis. It is difficult to understand and transform scenes appearance and semantic information from another view, thus we use deformed convolution in the U-net network to improve the network's ability to extract features of objects at different scales.
  arXiv  Detail & Related papers  (2020-07-20T03:08:36Z)
• Seeing eye-to-eye? A comparison of object recognition performance in humans and deep convolutional neural networks under image manipulation [0.0]
  This study aims towards a behavioral comparison of visual core object recognition performance between humans and feedforward neural networks. Analyses of accuracy revealed that humans not only outperform DCNNs on all conditions, but also display significantly greater robustness towards shape and most notably color alterations.
  arXiv  Detail & Related papers  (2020-07-13T10:26:30Z)
• Intrinsic Autoencoders for Joint Neural Rendering and Intrinsic Image Decomposition [67.9464567157846]
  We propose an autoencoder for joint generation of realistic images from synthetic 3D models while simultaneously decomposing real images into their intrinsic shape and appearance properties. Our experiments confirm that a joint treatment of rendering and decomposition is indeed beneficial and that our approach outperforms state-of-the-art image-to-image translation baselines both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2020-06-29T12:53:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.