Hierarchical Fine-Grained Image Forgery Detection and Localization
- URL: http://arxiv.org/abs/2303.17111v1
- Date: Thu, 30 Mar 2023 02:51:52 GMT
- Title: Hierarchical Fine-Grained Image Forgery Detection and Localization
- Authors: Xiao Guo, Xiaohong Liu, Zhiyuan Ren, Steven Grosz, Iacopo Masi,
Xiaoming Liu
- Abstract summary: We present a hierarchical fine-grained formulation for IFDL representation learning.
We first represent forgery attributes of a manipulated image with multiple labels at different levels.
As a result, the algorithm is encouraged to learn both comprehensive features and inherent hierarchical nature of different forgery attributes.
- Score: 24.595585815686007
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Differences in forgery attributes of images generated in CNN-synthesized and
image-editing domains are large, and such differences make a unified image
forgery detection and localization (IFDL) challenging. To this end, we present
a hierarchical fine-grained formulation for IFDL representation learning.
Specifically, we first represent forgery attributes of a manipulated image with
multiple labels at different levels. Then we perform fine-grained
classification at these levels using the hierarchical dependency between them.
As a result, the algorithm is encouraged to learn both comprehensive features
and inherent hierarchical nature of different forgery attributes, thereby
improving the IFDL representation. Our proposed IFDL framework contains three
components: multi-branch feature extractor, localization and classification
modules. Each branch of the feature extractor learns to classify forgery
attributes at one level, while localization and classification modules segment
the pixel-level forgery region and detect image-level forgery, respectively.
Lastly, we construct a hierarchical fine-grained dataset to facilitate our
study. We demonstrate the effectiveness of our method on $7$ different
benchmarks, for both tasks of IFDL and forgery attribute classification. Our
source code and dataset can be found:
\href{https://github.com/CHELSEA234/HiFi_IFDL}{github.com/CHELSEA234/HiFi-IFDL}.
Related papers
- Hierarchical Matching and Reasoning for Multi-Query Image Retrieval [113.44470784756308]
We propose a novel Hierarchical Matching and Reasoning Network (HMRN) for Multi-Query Image Retrieval (MQIR)
It disentangles MQIR into three hierarchical semantic representations, which is responsible to capture fine-grained local details, contextual global scopes, and high-level inherent correlations.
Our HMRN substantially surpasses the current state-of-the-art methods.
arXiv Detail & Related papers (2023-06-26T07:03:56Z) - A Capsule Network for Hierarchical Multi-Label Image Classification [2.507647327384289]
Hierarchical multi-label classification applies when a multi-class image classification problem is arranged into smaller ones based upon a hierarchy or taxonomy.
We propose a multi-label capsule network (ML-CapsNet) for hierarchical classification.
arXiv Detail & Related papers (2022-09-13T04:17:08Z) - Attribute Group Editing for Reliable Few-shot Image Generation [85.52840521454411]
We propose a new editing-based method, i.e., Attribute Group Editing (AGE), for few-shot image generation.
AGE examines the internal representation learned in GANs and identifies semantically meaningful directions.
arXiv Detail & Related papers (2022-03-16T06:54:09Z) - DSNet: A Dual-Stream Framework for Weakly-Supervised Gigapixel Pathology
Image Analysis [78.78181964748144]
We present a novel weakly-supervised framework for classifying whole slide images (WSIs)
WSIs are commonly processed by patch-wise classification with patch-level labels.
With image-level labels only, patch-wise classification would be sub-optimal due to inconsistency between the patch appearance and image-level label.
arXiv Detail & Related papers (2021-09-13T09:10:43Z) - Semantic Distribution-aware Contrastive Adaptation for Semantic
Segmentation [50.621269117524925]
Domain adaptive semantic segmentation refers to making predictions on a certain target domain with only annotations of a specific source domain.
We present a semantic distribution-aware contrastive adaptation algorithm that enables pixel-wise representation alignment.
We evaluate SDCA on multiple benchmarks, achieving considerable improvements over existing algorithms.
arXiv Detail & Related papers (2021-05-11T13:21:25Z) - Re-rank Coarse Classification with Local Region Enhanced Features for
Fine-Grained Image Recognition [22.83821575990778]
We re-rank the TopN classification results by using the local region enhanced embedding features to improve the Top1 accuracy.
To learn more effective semantic global features, we design a multi-level loss over an automatically constructed hierarchical category structure.
Our method achieves state-of-the-art performance on three benchmarks: CUB-200-2011, Stanford Cars, and FGVC Aircraft.
arXiv Detail & Related papers (2021-02-19T11:30:25Z) - Zero-Shot Recognition through Image-Guided Semantic Classification [9.291055558504588]
We present a new embedding-based framework for zero-shot learning (ZSL)
Motivated by the binary relevance method for multi-label classification, we propose to inversely learn the mapping between an image and a semantic classifier.
IGSC is conceptually simple and can be realized by a slight enhancement of an existing deep architecture for classification.
arXiv Detail & Related papers (2020-07-23T06:22:40Z) - A Universal Representation Transformer Layer for Few-Shot Image
Classification [43.31379752656756]
Few-shot classification aims to recognize unseen classes when presented with only a small number of samples.
We consider the problem of multi-domain few-shot image classification, where unseen classes and examples come from diverse data sources.
Here, we propose a Universal Representation Transformer layer, that meta-learns to leverage universal features for few-shot classification.
arXiv Detail & Related papers (2020-06-21T03:08:00Z) - Hierarchical Image Classification using Entailment Cone Embeddings [68.82490011036263]
We first inject label-hierarchy knowledge into an arbitrary CNN-based classifier.
We empirically show that availability of such external semantic information in conjunction with the visual semantics from images boosts overall performance.
arXiv Detail & Related papers (2020-04-02T10:22:02Z) - Cross-Domain Few-Shot Classification via Learned Feature-Wise
Transformation [109.89213619785676]
Few-shot classification aims to recognize novel categories with only few labeled images in each class.
Existing metric-based few-shot classification algorithms predict categories by comparing the feature embeddings of query images with those from a few labeled images.
While promising performance has been demonstrated, these methods often fail to generalize to unseen domains.
arXiv Detail & Related papers (2020-01-23T18:55:43Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.