ForensicsTok: Forensics-Guided Tokenized Modeling for Image Tampering Localization

• URL: http://arxiv.org/abs/2606.24538v1
• Date: Tue, 23 Jun 2026 13:06:18 GMT
• Title: ForensicsTok: Forensics-Guided Tokenized Modeling for Image Tampering Localization
• Abstract summary: We propose ForensicsTok, which reformulates image manipulation localization as an autoregressive sequence generation task.<n>ForensicsTok directly generates spatially grounded token sequences, enabling precise mask prediction without intermediary supervision.
• Score: 31.83661814822312
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-modal Large Language Models (MLLMs) offer powerful reasoning for forensic tasks, yet existing approaches utilizing exogenous segmentation decoders often suffer from suboptimal localization. The reliance on stitched pipelines introduces information bottlenecks during backpropagation, which dilutes spatial signals and is limited by semantic priors of the segmentor. To address these limitations, we propose ForensicsTok, which reformulates image manipulation localization as an autoregressive sequence generation task. ForensicsTok directly generates spatially grounded token sequences, enabling precise mask prediction without intermediary supervision. Specifically, we introduce a Token Splatting Decoder (TSD) to map tokens to binary masks via codebook-aware code smoothing, which mitigates sharp gradients from deterministic detokenizers. Furthermore, to capture diverse tampering clues, we propose a Hierarchical Expert Fusion (HEF) module that injects multi-scale features from a forensic expert model. This unified architecture effectively compensates for the lack of forensic priors in standard MLLMs. Extensive experiments on six benchmarks show that ForensicsTok substantially improves over existing MLLM-based baselines and slightly improves over strong forensic expert baselines, while exhibiting stronger robustness to perturbations.

Related papers

• PATE-Forensics: Perception-as-Tool for Explainable Deepfake Forensics with General-Purpose MLLMs [27.658161939360202]
  PATE-Forensics architecturally decouples detection and localization from explanation generation.<n>On DDL-X Track 3, PATE-Forensics achieves the best official score of 0.89, outperforming the second-ranked team by 0.19 points.
  arXiv  Detail & Related papers  (2026-08-19T06:10:00Z)
• Deep Residual Injection for Full-Spectrum Forensic Signal Perception in Multimodal Large Language Models [51.099179436350376]
  Multimodal large language models (MLLMs) have been increasingly adopted in forensics for their robust semantic understanding.<n>We propose Deep Visual Residual MLLM (Deep-VRM) to preserve early semantic processing while injecting artifact-specific visual signals as a residual path into an intermediate layer.<n>This enables later layers to jointly model semantic reasoning and signal-level forensic cues, and surprisingly, the model learns to adaptively leverage different levels of forensic signals depending on the input.
  arXiv  Detail & Related papers  (2026-06-14T16:05:45Z)
• Excite, Attend and Segment (EASe): Domain-Agnostic Fine-Grained Mask Discovery with Feature Calibration and Self-Supervised Upsampling [0.0]
  Excite, Attend and Segment (EASe) is an unsupervised domain-agnostic semantic segmentation framework.<n>Our evaluation demonstrates superior performance of EASe over previous state-of-the-arts.
  arXiv  Detail & Related papers  (2026-03-31T22:03:15Z)
• ForensicZip: More Tokens are Better but Not Necessary in Forensic Vision-Language Models [108.5042835056188]
  We introduce ForensicZip, a training-free framework that reformulates token compression from a forgery-driven perspective.<n>ForensicZip achieves $2.97times$ speedup and over 90% FLOPs reduction while maintaining state-of-the-art detection performance.
  arXiv  Detail & Related papers  (2026-03-12T17:30:49Z)
• SIGMAE: A Spectral-Index-Guided Foundation Model for Multispectral Remote Sensing [43.39478017496301]
  Masked Autoencoder (MAE)-based pretraining stands out for its strong capability to learn general feature representations via reconstructing masked image regions.<n>Applying MAE to multispectral remote sensing images remains challenging due to complex backgrounds, indistinct targets, and the lack of semantic guidance during masking.<n> SIGMAE incorporates domain-specific spectral indices as prior knowledge to guide dynamic token masking toward informative regions.
  arXiv  Detail & Related papers  (2026-03-08T04:55:41Z)
• Accelerate Speculative Decoding with Sparse Computation in Verification [49.74839681322316]
  Speculative decoding accelerates autoregressive language model inference by verifying multiple draft tokens in parallel.<n>Existing sparsification methods are designed primarily for standard token-by-token autoregressive decoding.<n>We propose a sparse verification framework that jointly sparsifies attention, FFN, and MoE components during the verification stage to reduce the dominant computation cost.
  arXiv  Detail & Related papers  (2025-12-26T07:53:41Z)
• Propose and Rectify: A Forensics-Driven MLLM Framework for Image Manipulation Localization [49.71303998618939]
  This paper presents a novel Propose-Rectify framework that bridges semantic reasoning with forensic-specific analysis.<n>Our framework ensures that initial semantic proposals are systematically validated and enhanced through concrete technical evidence, resulting in comprehensive detection accuracy and localization precision.
  arXiv  Detail & Related papers  (2025-08-25T12:43:53Z)
• ForgeryGPT: Multimodal Large Language Model For Explainable Image Forgery Detection and Localization [49.12958154544838]
  ForgeryGPT is a novel framework that advances the Image Forgery Detection and localization task.<n>It captures high-order correlations of forged images from diverse linguistic feature spaces.<n>It enables explainable generation and interactive dialogue through a newly customized Large Language Model (LLM) architecture.
  arXiv  Detail & Related papers  (2024-10-14T07:56:51Z)
• Mixture-of-Noises Enhanced Forgery-Aware Predictor for Multi-Face Manipulation Detection and Localization [52.87635234206178]
  This paper proposes a new framework, namely MoNFAP, specifically tailored for multi-face manipulation detection and localization. The framework incorporates two novel modules: the Forgery-aware Unified Predictor (FUP) Module and the Mixture-of-Noises Module (MNM)
  arXiv  Detail & Related papers  (2024-08-05T08:35:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.