Exploring Saliency Bias in Manipulation Detection

• URL: http://arxiv.org/abs/2402.07338v3
• Date: Tue, 20 Aug 2024 18:20:44 GMT
• Title: Exploring Saliency Bias in Manipulation Detection
• Authors: Joshua Krinsky, Alan Bettis, Qiuyu Tang, Daniel Moreira, Aparna Bharati,
• Abstract summary: Social media-fuelled explosion of fake news and misinformation supported by tampered images has led to growth in the development of models and datasets for image manipulation detection. Existing detection methods mostly treat media objects in isolation, without considering the impact of specific manipulations on viewer perception. We propose a framework to analyze the trends of visual and semantic saliency in popular image manipulation datasets and their impact on detection.
• Score: 2.156234249946792
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The social media-fuelled explosion of fake news and misinformation supported by tampered images has led to growth in the development of models and datasets for image manipulation detection. However, existing detection methods mostly treat media objects in isolation, without considering the impact of specific manipulations on viewer perception. Forensic datasets are usually analyzed based on the manipulation operations and corresponding pixel-based masks, but not on the semantics of the manipulation, i.e., type of scene, objects, and viewers' attention to scene content. The semantics of the manipulation play an important role in spreading misinformation through manipulated images. In an attempt to encourage further development of semantic-aware forensic approaches to understand visual misinformation, we propose a framework to analyze the trends of visual and semantic saliency in popular image manipulation datasets and their impact on detection.

Related papers

• Deepfake detection by exploiting surface anomalies: the SurFake approach [29.088218634944116]
  This paper investigates how deepfake creation can impact on the characteristics that the whole scene had at the time of the acquisition. By resorting to the analysis of the characteristics of the surfaces depicted in the image it is possible to obtain a descriptor usable to train a CNN for deepfake detection.
  arXiv  Detail & Related papers  (2023-10-31T16:54:14Z)
• AutoSplice: A Text-prompt Manipulated Image Dataset for Media Forensics [31.714342131823987]
  This paper aims to investigate the level of challenge that language-image generation models pose to media forensics. We propose a new approach that leverages the DALL-E2 language-image model to automatically generate and splice masked regions guided by a text prompt. This approach has resulted in the creation of a new image dataset called AutoSplice, containing 5,894 manipulated and authentic images.
  arXiv  Detail & Related papers  (2023-04-14T00:14:08Z)
• Fighting Malicious Media Data: A Survey on Tampering Detection and Deepfake Detection [115.83992775004043]
  Recent advances in deep learning, particularly deep generative models, open the doors for producing perceptually convincing images and videos at a low cost. This paper provides a comprehensive review of the current media tampering detection approaches, and discusses the challenges and trends in this field for future research.
  arXiv  Detail & Related papers  (2022-12-12T02:54:08Z)
• DisPositioNet: Disentangled Pose and Identity in Semantic Image Manipulation [83.51882381294357]
  DisPositioNet is a model that learns a disentangled representation for each object for the task of image manipulation using scene graphs. Our framework enables the disentanglement of the variational latent embeddings as well as the feature representation in the graph.
  arXiv  Detail & Related papers  (2022-11-10T11:47:37Z)
• ObjectFormer for Image Manipulation Detection and Localization [118.89882740099137]
  We propose ObjectFormer to detect and localize image manipulations. We extract high-frequency features of the images and combine them with RGB features as multimodal patch embeddings. We conduct extensive experiments on various datasets and the results verify the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2022-03-28T12:27:34Z)
• Complex Scene Image Editing by Scene Graph Comprehension [17.72638225034884]
  We propose a two-stage method for achieving complex scene image editing by Scene Graph (SGC-Net) In the first stage, we train a Region of Interest (RoI) prediction network that uses scene graphs and predict the locations of the target objects. The second stage uses a conditional diffusion model to edit the image based on our RoI predictions.
  arXiv  Detail & Related papers  (2022-03-24T05:12:54Z)
• Detect and Locate: A Face Anti-Manipulation Approach with Semantic and Noise-level Supervision [67.73180660609844]
  We propose a conceptually simple but effective method to efficiently detect forged faces in an image. The proposed scheme relies on a segmentation map that delivers meaningful high-level semantic information clues about the image. The proposed model achieves state-of-the-art detection accuracy and remarkable localization performance.
  arXiv  Detail & Related papers  (2021-07-13T02:59:31Z)
• Content-Based Detection of Temporal Metadata Manipulation [91.34308819261905]
  We propose an end-to-end approach to verify whether the purported time of capture of an image is consistent with its content and geographic location. The central idea is the use of supervised consistency verification, in which we predict the probability that the image content, capture time, and geographical location are consistent. Our approach improves upon previous work on a large benchmark dataset, increasing the classification accuracy from 59.03% to 81.07%.
  arXiv  Detail & Related papers  (2021-03-08T13:16:19Z)
• Self-supervised Segmentation via Background Inpainting [96.10971980098196]
  We introduce a self-supervised detection and segmentation approach that can work with single images captured by a potentially moving camera. We exploit a self-supervised loss function that we exploit to train a proposal-based segmentation network. We apply our method to human detection and segmentation in images that visually depart from those of standard benchmarks and outperform existing self-supervised methods.
  arXiv  Detail & Related papers  (2020-11-11T08:34:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.