Modelship Attribution: Tracing Multi-Stage Manipulations Across Generative Models

• URL: http://arxiv.org/abs/2506.02405v1
• Date: Tue, 03 Jun 2025 03:45:09 GMT
• Title: Modelship Attribution: Tracing Multi-Stage Manipulations Across Generative Models
• Authors: Zhiya Tan, Xin Zhang, Joey Tianyi Zhou,
• Abstract summary: "Modelship Attribution" aims to trace the evolution of manipulated images by identifying the generative models involved and reconstructing the sequence of edits they performed.<n>We introduce the modelship attribution transformer (MAT), a framework designed to effectively recognize and attribute the contributions of various models within complex, multi-stage manipulation.
• Score: 37.368187232084324
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As generative techniques become increasingly accessible, authentic visuals are frequently subjected to iterative alterations by various individuals employing a variety of tools. Currently, to avoid misinformation and ensure accountability, a lot of research on detection and attribution is emerging. Although these methods demonstrate promise in single-stage manipulation scenarios, they fall short when addressing complex real-world iterative manipulation. In this paper, we are the first, to the best of our knowledge, to systematically model this real-world challenge and introduce a novel method to solve it. We define a task called "Modelship Attribution", which aims to trace the evolution of manipulated images by identifying the generative models involved and reconstructing the sequence of edits they performed. To realistically simulate this scenario, we utilize three generative models, StyleMapGAN, DiffSwap, and FacePartsSwap, that sequentially modify distinct regions of the same image. This process leads to the creation of the first modelship dataset, comprising 83,700 images (16,740 images*5). Given that later edits often overwrite the fingerprints of earlier models, the focus shifts from extracting blended fingerprints to characterizing each model's distinctive editing patterns. To tackle this challenge, we introduce the modelship attribution transformer (MAT), a purpose-built framework designed to effectively recognize and attribute the contributions of various models within complex, multi-stage manipulation workflows. Through extensive experiments and comparative analysis with other related methods, our results, including comprehensive ablation studies, demonstrate that the proposed approach is a highly effective solution for modelship attribution.

Related papers

• What Changed? Detecting and Evaluating Instruction-Guided Image Edits with Multimodal Large Language Models [88.398085358514]
  DICE is a model designed to detect localized differences between the original and the edited image.<n>It is trained using a strategy that leverages self-supervision, distillation from inpainting networks, and full supervision.<n>We demonstrate that DICE effectively identifies coherent edits, effectively evaluating images generated by different editing models with a strong correlation with human judgment.
  arXiv  Detail & Related papers  (2025-05-26T18:00:10Z)
• EditAR: Unified Conditional Generation with Autoregressive Models [58.093860528672735]
  We propose EditAR, a single unified autoregressive framework for a variety of conditional image generation tasks.<n>The model takes both images and instructions as inputs, and predicts the edited images tokens in a vanilla next-token paradigm.<n>We evaluate its effectiveness across diverse tasks on established benchmarks, showing competitive performance to various state-of-the-art task-specific methods.
  arXiv  Detail & Related papers  (2025-01-08T18:59:35Z)
• A Simple Approach to Unifying Diffusion-based Conditional Generation [63.389616350290595]
  We introduce a simple, unified framework to handle diverse conditional generation tasks.<n>Our approach enables versatile capabilities via different inference-time sampling schemes.<n>Our model supports additional capabilities like non-spatially aligned and coarse conditioning.
  arXiv  Detail & Related papers  (2024-10-15T09:41:43Z)
• Model Attribution in LLM-Generated Disinformation: A Domain Generalization Approach with Supervised Contrastive Learning [26.02988481241285]
  Modern large language models (LLMs) produce disinformation with human-like quality. diversity in prompting methods used to generate disinformation complicates accurate source attribution. We introduce the concept of model attribution as a domain generalization problem.
  arXiv  Detail & Related papers  (2024-07-31T00:56:09Z)
• Has Your Pretrained Model Improved? A Multi-head Posterior Based Approach [25.927323251675386]
  We leverage the meta-features associated with each entity as a source of worldly knowledge and employ entity representations from the models. We propose using the consistency between these representations and the meta-features as a metric for evaluating pre-trained models. Our method's effectiveness is demonstrated across various domains, including models with relational datasets, large language models and image models.
  arXiv  Detail & Related papers  (2024-01-02T17:08:26Z)
• Model Synthesis for Zero-Shot Model Attribution [26.835046772924258]
  generative models are shaping various fields such as art, design, and human-computer interaction.<n>We propose a model synthesis technique, which generates numerous synthetic models mimicking the fingerprint patterns of real-world generative models.<n>Our experiments demonstrate that this fingerprint extractor, trained solely on synthetic models, achieves impressive zero-shot generalization on a wide range of real-world generative models.
  arXiv  Detail & Related papers  (2023-07-29T13:00:42Z)
• IRGen: Generative Modeling for Image Retrieval [82.62022344988993]
  In this paper, we present a novel methodology, reframing image retrieval as a variant of generative modeling. We develop our model, dubbed IRGen, to address the technical challenge of converting an image into a concise sequence of semantic units. Our model achieves state-of-the-art performance on three widely-used image retrieval benchmarks and two million-scale datasets.
  arXiv  Detail & Related papers  (2023-03-17T17:07:36Z)
• Learning Robust Representations Of Generative Models Using Set-Based Artificial Fingerprints [14.191129493685212]
  Existing methods approximate the distance between the models via their sample distributions. We consider unique traces (a.k.a. "artificial fingerprints") as representations of generative models. We propose a new learning method based on set-encoding and contrastive training.
  arXiv  Detail & Related papers  (2022-06-04T23:20:07Z)
• Learning to Model Editing Processes [98.11448946134894]
  We propose modeling editing processes, modeling the whole process of iteratively generating sequences. We form a conceptual framework to describe the likelihood of multi-step edits, and describe neural models that can learn a generative model of sequences based on these multistep edits.
  arXiv  Detail & Related papers  (2022-05-24T21:32:52Z)
• End-to-End Visual Editing with a Generatively Pre-Trained Artist [78.5922562526874]
  We consider the targeted image editing problem: blending a region in a source image with a driver image that specifies the desired change. We propose a self-supervised approach that simulates edits by augmenting off-the-shelf images in a target domain. We show that different blending effects can be learned by an intuitive control of the augmentation process, with no other changes required to the model architecture.
  arXiv  Detail & Related papers  (2022-05-03T17:59:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.