SUMI-IFL: An Information-Theoretic Framework for Image Forgery Localization with Sufficiency and Minimality Constraints

• URL: http://arxiv.org/abs/2412.09981v2
• Date: Fri, 27 Dec 2024 05:58:54 GMT
• Title: SUMI-IFL: An Information-Theoretic Framework for Image Forgery Localization with Sufficiency and Minimality Constraints
• Authors: Ziqi Sheng, Wei Lu, Xiangyang Luo, Jiantao Zhou, Xiaochun Cao,
• Abstract summary: Image forgery localization (IFL) is a crucial technique for preventing tampered image misuse and protecting social safety. We introduce a novel information-theoretic IFL framework named SUMI-IFL that imposes sufficiency-view and minimality-view constraints on forgery feature representation.
• Score: 66.85363924364628
• License:
• Abstract: Image forgery localization (IFL) is a crucial technique for preventing tampered image misuse and protecting social safety. However, due to the rapid development of image tampering technologies, extracting more comprehensive and accurate forgery clues remains an urgent challenge. To address these challenges, we introduce a novel information-theoretic IFL framework named SUMI-IFL that imposes sufficiency-view and minimality-view constraints on forgery feature representation. First, grounded in the theoretical analysis of mutual information, the sufficiency-view constraint is enforced on the feature extraction network to ensure that the latent forgery feature contains comprehensive forgery clues. Considering that forgery clues obtained from a single aspect alone may be incomplete, we construct the latent forgery feature by integrating several individual forgery features from multiple perspectives. Second, based on the information bottleneck, the minimality-view constraint is imposed on the feature reasoning network to achieve an accurate and concise forgery feature representation that counters the interference of task-unrelated features. Extensive experiments show the superior performance of SUMI-IFL to existing state-of-the-art methods, not only on in-dataset comparisons but also on cross-dataset comparisons.

Related papers

• Information Sifting Funnel: Privacy-preserving Collaborative Inference Against Model Inversion Attacks [9.092229145160763]
  complexity of neural networks and inference tasks poses significant challenges for resource-constrained edge devices. Collaborative inference mitigates this by assigning shallow feature extraction to edge devices and offloading features to the cloud for further inference. We propose SiftFunnel, a privacy-preserving framework for collaborative inference.
  arXiv  Detail & Related papers  (2025-01-01T13:00:01Z)
• FFAA: Multimodal Large Language Model based Explainable Open-World Face Forgery Analysis Assistant [59.2438504610849]
  We introduce FFAA: Face Forgery Analysis Assistant, consisting of a fine-tuned Multimodal Large Language Model (MLLM) and Multi-answer Intelligent Decision System (MIDS) Our method not only provides user-friendly and explainable results but also significantly boosts accuracy and robustness compared to previous methods.
  arXiv  Detail & Related papers  (2024-08-19T15:15:20Z)
• Small Object Detection via Coarse-to-fine Proposal Generation and Imitation Learning [52.06176253457522]
  We propose a two-stage framework tailored for small object detection based on the Coarse-to-fine pipeline and Feature Imitation learning. CFINet achieves state-of-the-art performance on the large-scale small object detection benchmarks, SODA-D and SODA-A.
  arXiv  Detail & Related papers  (2023-08-18T13:13:09Z)
• Robust Saliency-Aware Distillation for Few-shot Fine-grained Visual Recognition [57.08108545219043]
  Recognizing novel sub-categories with scarce samples is an essential and challenging research topic in computer vision. Existing literature addresses this challenge by employing local-based representation approaches. This article proposes a novel model, Robust Saliency-aware Distillation (RSaD), for few-shot fine-grained visual recognition.
  arXiv  Detail & Related papers  (2023-05-12T00:13:17Z)
• Multi-view Feature Extraction based on Triple Contrastive Heads [1.4467794332678539]
  We propose a novel multi-view feature extraction method based on triple contrastive heads. The proposed method offers a strong advantage for multi-view feature extraction.
  arXiv  Detail & Related papers  (2023-03-22T14:56:51Z)
• Ontology-aware Network for Zero-shot Sketch-based Image Retrieval [6.40453257995913]
  Zero-Shot Sketch-Based Image Retrieval (ZSSBIR) is an emerging task. Recent work has begun to consider the triplet-based or contrast-based loss to mine inter-class information. An Ontology-Aware Network (OAN) is proposed to respond to these issues.
  arXiv  Detail & Related papers  (2023-02-20T15:44:41Z)
• Toward Certified Robustness Against Real-World Distribution Shifts [65.66374339500025]
  We train a generative model to learn perturbations from data and define specifications with respect to the output of the learned model. A unique challenge arising from this setting is that existing verifiers cannot tightly approximate sigmoid activations. We propose a general meta-algorithm for handling sigmoid activations which leverages classical notions of counter-example-guided abstraction refinement.
  arXiv  Detail & Related papers  (2022-06-08T04:09:13Z)
• A Principled Design of Image Representation: Towards Forensic Tasks [75.40968680537544]
  We investigate the forensic-oriented image representation as a distinct problem, from the perspectives of theory, implementation, and application. At the theoretical level, we propose a new representation framework for forensics, called Dense Invariant Representation (DIR), which is characterized by stable description with mathematical guarantees. We demonstrate the above arguments on the dense-domain pattern detection and matching experiments, providing comparison results with state-of-the-art descriptors.
  arXiv  Detail & Related papers  (2022-03-02T07:46:52Z)
• Mixed-supervised segmentation: Confidence maximization helps knowledge distillation [24.892332859630518]
  In this work, we propose a dual-branch architecture for deep neural networks. The upper branch (teacher) receives strong annotations, while the bottom one (student) is driven by limited supervision and guided by the upper branch. We show that the synergy between the entropy and KL divergence yields substantial improvements in performance.
  arXiv  Detail & Related papers  (2021-09-21T20:06:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.