When Recovery Matters: The Blind Spot of Surrogate Privacy in MLLM Editing

• URL: http://arxiv.org/abs/2606.07171v1
• Date: Fri, 05 Jun 2026 11:40:03 GMT
• Title: When Recovery Matters: The Blind Spot of Surrogate Privacy in MLLM Editing
• Abstract summary: We introduce SPPE (Surrogate-based Privacy-Preserving Editing), the first recovery-oriented benchmark covering 36 fine-grained privacy categories and 65 editing instructions.<n>For editability assessment, ERMA improves over the best-performing baselines by 13.9% in SRCC and 12.3% in PLCC.<n>Experiments on SPPE and InstructPix2Pix show consistent improvements on both tasks.
• Score: 61.80513991207956
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal Large Language Models (MLLMs) enable flexible instruction-driven image editing, but privacy risks arise when user images expose diverse and user-specific private content. Canonical privacy protection strategies typically substitute sensitive regions with surrogate content before cloud editing. Yet, the resulting output is often an edited surrogate rather than the desired edited source image, neglecting the local recovery in both design and evaluation scope. To this end, we introduce SPPE (Surrogate-based Privacy-Preserving Editing), the first recovery-oriented benchmark covering 36 fine-grained privacy categories and 65 editing instructions. It defines two complementary tasks: 1) editability assessment, which estimates before cloud interaction whether a surrogate can induce an edit consistent with the original image; and 2) surrogate-to-source edit recovery, which evaluates whether the edited surrogate can be transferred back to the private source with the edit effect preserved. We address each task with a dedicated method: ERMA predicts surrogate editability through instruction-aware multimodal relation modeling, while \method performs cycle-consistent recovery by using the surrogate editing pair as visual edit evidence and the source image as a source-preserving anchor. Experiments on SPPE and InstructPix2Pix show consistent improvements on both tasks. For editability assessment, ERMA improves over the best-performing baselines by 13.9% in SRCC and 12.3% in PLCC. For surrogate-to-source edit recovery, C2E-S2SER outperforms SOER across all 8 source integrity and edit consistency metrics on SPPE.

Related papers

• Can We Perform Online RL for Image Editing without Editing Rewards? [56.482951092646026]
  We argue that the standard image editing dimensions have potential to be mapped to the text-to-image (T2I) reward space.<n>We introduce Lever-Edit, a framework that learns a reward-aligned captioner for counterfactual target descriptions.<n>Experiments show competitive editing alignment and source preservation against editing-reward-based fine-tuning, while outperforming intuitive transfer baselines.
  arXiv  Detail & Related papers  (2026-08-24T04:08:52Z)
• Model the Edit, Not the Image: Visual Autoregressive Editing from a Source-Centric Perspective [23.408232115772833]
  Next-scale visual autoregressive models ( VARs) have emerged as a powerful generative paradigm, producing high-quality images through efficient coarse-to-fine prediction.<n>We propose textbfEditMod, which compares source- and target-conditioned predictions under a shared autoregressive context.<n>Experiments show that EditMod achieves leading source-image fidelity while maintaining strong text alignment, and completes end-to-end editing of a 1K image in only 1.57 seconds on a single A100 GPU.
  arXiv  Detail & Related papers  (2026-08-10T03:05:09Z)
• GMO-E$^2$DIT: Grounded Multi-Operation Editing for E-Commerce Images [16.213122519512932]
  Real-world e-commerce image editing often requires multiple, localized, and auditable operations rather than global restyling.<n>Existing one-shot editors conflate intent resolution, spatial grounding, and synthesis into a single step.<n>We introduce GMO-E$2$DIT, an agentic editing framework that couples a Vision-Language Model (VLM) with a mask-conditioned image editor to tackle structured multi-turn task completion.
  arXiv  Detail & Related papers  (2026-07-01T13:23:45Z)
• CoEditor++: Instruction-based Visual Editing via Cognitive Reasoning [98.98349220451216]
  CoEditor++ is a training-free framework that decomposes editing into "what to edit" and "how to edit"<n>We show that CoEditor++ achieves state-of-the-art performance in both general editing and responsible editing tasks.<n>Our findings suggest the potential toward cognitive-centric instruction-based image editing.
  arXiv  Detail & Related papers  (2026-01-31T12:20:46Z)
• ProEdit: Inversion-based Editing From Prompts Done Right [63.554692704101]
  Inversion-based visual editing provides an effective and training-free way to edit an image or a video based on user instructions.<n>Existing methods typically inject source image information during the sampling process to maintain editing consistency.<n>We propose ProEdit to address this issue both in the attention and the latent aspects.
  arXiv  Detail & Related papers  (2025-12-26T18:59:14Z)
• Lost in Edits? A $λ$-Compass for AIGC Provenance [119.95562081325552]
  We propose a novel latent-space attribution method that robustly identifies and differentiates authentic outputs from manipulated ones.<n>LambdaTracer is effective across diverse iterative editing processes, whether automated through text-guided editing tools such as InstructPix2Pix or performed manually with editing software such as Adobe Photoshop.
  arXiv  Detail & Related papers  (2025-02-05T06:24:25Z)
• UIP2P: Unsupervised Instruction-based Image Editing via Edit Reversibility Constraint [87.20985852686785]
  We propose an unsupervised instruction-based image editing approach that removes the need for ground-truth edited images during training.<n>Our approach addresses these challenges by introducing a novel editing mechanism called Edit Reversibility Constraint (ERC), which applies forward and reverse edits in one training step.<n>This allows us to bypass the need for ground-truth edited images and unlock training for the first time on datasets comprising either real image-caption pairs or image-caption-instruction triplets.
  arXiv  Detail & Related papers  (2024-12-19T18:59:58Z)
• E4C: Enhance Editability for Text-Based Image Editing by Harnessing Efficient CLIP Guidance [13.535394339438428]
  Diffusion-based image editing is a composite process of preserving the source image content and generating new content or applying modifications. We propose a zero-shot image editing method, named textbfEnhance textbfEditability for text-based image textbfEditing via textbfCLIP guidance.
  arXiv  Detail & Related papers  (2024-03-15T09:26:48Z)
• Object-aware Inversion and Reassembly for Image Editing [61.19822563737121]
  We propose Object-aware Inversion and Reassembly (OIR) to enable object-level fine-grained editing. We use our search metric to find the optimal inversion step for each editing pair when editing an image. Our method achieves superior performance in editing object shapes, colors, materials, categories, etc., especially in multi-object editing scenarios.
  arXiv  Detail & Related papers  (2023-10-18T17:59:02Z)
• Eliminating Contextual Prior Bias for Semantic Image Editing via Dual-Cycle Diffusion [35.95513392917737]
  A novel approach called Dual-Cycle Diffusion generates an unbiased mask to guide image editing. Our experiments demonstrate the effectiveness of the proposed method, as it significantly improves the D-CLIP score from 0.272 to 0.283.
  arXiv  Detail & Related papers  (2023-02-05T14:30:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.