RefineAnything: Multimodal Region-Specific Refinement for Perfect Local Details

• URL: http://arxiv.org/abs/2604.06870v1
• Date: Wed, 08 Apr 2026 09:32:15 GMT
• Title: RefineAnything: Multimodal Region-Specific Refinement for Perfect Local Details
• Abstract summary: We introduce region-specific image refinement as a dedicated problem setting.<n>The goal is to restore fine-grained details while keeping all non-edited pixels strictly unchanged.<n>Existing instruction-driven editing models emphasize coarse-grained semantic edits.
• Score: 41.48256151708512
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce region-specific image refinement as a dedicated problem setting: given an input image and a user-specified region (e.g., a scribble mask or a bounding box), the goal is to restore fine-grained details while keeping all non-edited pixels strictly unchanged. Despite rapid progress in image generation, modern models still frequently suffer from local detail collapse (e.g., distorted text, logos, and thin structures). Existing instruction-driven editing models emphasize coarse-grained semantic edits and often either overlook subtle local defects or inadvertently change the background, especially when the region of interest occupies only a small portion of a fixed-resolution input. We present RefineAnything, a multimodal diffusion-based refinement model that supports both reference-based and reference-free refinement. Building on a counter-intuitive observation that crop-and-resize can substantially improve local reconstruction under a fixed VAE input resolution, we propose Focus-and-Refine, a region-focused refinement-and-paste-back strategy that improves refinement effectiveness and efficiency by reallocating the resolution budget to the target region, while a blended-mask paste-back guarantees strict background preservation. We further introduce a boundary-aware Boundary Consistency Loss to reduce seam artifacts and improve paste-back naturalness. To support this new setting, we construct Refine-30K (20K reference-based and 10K reference-free samples) and introduce RefineEval, a benchmark that evaluates both edited-region fidelity and background consistency. On RefineEval, RefineAnything achieves strong improvements over competitive baselines and near-perfect background preservation, establishing a practical solution for high-precision local refinement. Project Page: https://limuloo.github.io/RefineAnything/.

Related papers

• HierEdit: Region-Aware Hierarchical Diffusion for Efficient High-Resolution Editing [83.1290629939693]
  High-resolution image editing is essential for professional and creative applications.<n>Current approaches redundantly process the entire image canvas or rely on large-scale high-resolution datasets.<n>We introduce HierEdit, a region-aware hierarchical diffusion framework for efficient and scalable high-resolution image editing.
  arXiv  Detail & Related papers  (2026-05-17T07:14:15Z)
• ResetEdit: Precise Text-guided Editing of Generated Image via Resettable Starting Latent [45.3757356193519]
  Recent advances in diffusion models have enabled high-quality image generation, leading to increasing demand for post-generation editing.<n>We propose ResetEdit, a proactive diffusion editing framework that embeds recoverable latent information directly into the generation process.<n>Built upon Stable Diffusion, ResetEdit integrates seamlessly with existing tuning-free editing methods and consistently outperforms state-of-the-art baselines in both controllability and visual fidelity.
  arXiv  Detail & Related papers  (2026-04-28T02:05:08Z)
• Region-Constrained Group Relative Policy Optimization for Flow-Based Image Editing [2.096755686662369]
  We propose RC-GRPO-Editing, a region-constrained GRPO post-training framework for flow-based image editing under deterministic ODE sampling.<n>It suppresses background-induced nuisance variance to enable cleaner localized credit assignment, improving region instruction adherence and preserving non-target content.<n> Experiments on CompBench show consistent improvements in editing region instruction adherence and non-target preservation.
  arXiv  Detail & Related papers  (2026-04-10T14:58:15Z)
• Mining Forgery Traces from Reconstruction Error: A Weakly Supervised Framework for Multimodal Deepfake Temporal Localization [25.609186178759995]
  We propose a weakly supervised forgery-based temporal localization framework that identifies forgeries via reconstruction errors.<n>Our framework uses a Masked Autoencoder (MAE) trained exclusively on authentic data to learn intrinsic its patterns.<n>We show that RT-Deepfake achieves state-of-the-art performance in weakly-supervised temporal localization.
  arXiv  Detail & Related papers  (2026-01-29T09:35:27Z)
• OmniRefiner: Reinforcement-Guided Local Diffusion Refinement [10.329465965964571]
  VAE-based latent compression discards subtle texture information, causing identity- and attribute-specific cues to vanish.<n>We introduce ourMthd, a detail-aware refinement framework that performs two consecutive stages of reference-driven correction.<n>Experiments demonstrate that ourMthd significantly improves reference alignment and fine-grained detail preservation.
  arXiv  Detail & Related papers  (2025-11-25T06:57:49Z)
• GaussianLens: Localized High-Resolution Reconstruction via On-Demand Gaussian Densification [77.40235389999]
  We propose a generalizable network that densifies the initial 3DGS to capture fine details in a user-specified local region of interest.<n>Experiments demonstrate our method's superior performance in local fine detail reconstruction and strong scalability to images of up to $1024times1024$ resolution.
  arXiv  Detail & Related papers  (2025-09-29T23:58:49Z)
• Uniform Attention Maps: Boosting Image Fidelity in Reconstruction and Editing [66.48853049746123]
  We analyze reconstruction from a structural perspective and propose a novel approach that replaces traditional cross-attention with uniform attention maps.<n>Our method effectively minimizes distortions caused by varying text conditions during noise prediction.<n> Experimental results demonstrate that our approach not only excels in achieving high-fidelity image reconstruction but also performs robustly in real image composition and editing scenarios.
  arXiv  Detail & Related papers  (2024-11-29T12:11:28Z)
• Coherent and Multi-modality Image Inpainting via Latent Space Optimization [61.99406669027195]
  PILOT (intextbfPainting vtextbfIa textbfLatent textbfOptextbfTimization) is an optimization approach grounded on a novel textitsemantic centralization and textitbackground preservation loss. Our method searches latent spaces capable of generating inpainted regions that exhibit high fidelity to user-provided prompts while maintaining coherence with the background.
  arXiv  Detail & Related papers  (2024-07-10T19:58:04Z)
• Building Bridges across Spatial and Temporal Resolutions: Reference-Based Super-Resolution via Change Priors and Conditional Diffusion Model [13.368558322546784]
  RefSR has the potential to build bridges across spatial and temporal resolutions of remote sensing images. Conditional diffusion models have opened up new opportunities for generating realistic high-resolution images. We propose Ref-Diff for RefSR, using the land cover change priors to guide the denoising process explicitly.
  arXiv  Detail & Related papers  (2024-03-26T07:48:49Z)
• LIME: Localized Image Editing via Attention Regularization in Diffusion Models [69.33072075580483]
  This paper introduces LIME for localized image editing in diffusion models.<n>LIME does not require user-specified regions of interest (RoI) or additional text input, but rather employs features from pre-trained methods and a straightforward clustering method to obtain precise editing mask.<n>We propose a novel cross-attention regularization technique that penalizes unrelated cross-attention scores in the RoI during the denoising steps, ensuring localized edits.
  arXiv  Detail & Related papers  (2023-12-14T18:59:59Z)
• GLocal: Global Graph Reasoning and Local Structure Transfer for Person Image Generation [2.580765958706854]
  We focus on person image generation, namely, generating person image under various conditions, e.g., corrupted texture or different pose. We present a GLocal framework to improve the occlusion-aware texture estimation by globally reasoning the style inter-correlations among different semantic regions. For local structural information preservation, we further extract the local structure of the source image and regain it in the generated image via local structure transfer.
  arXiv  Detail & Related papers  (2021-12-01T03:54:30Z)
• Image Fine-grained Inpainting [89.17316318927621]
  We present a one-stage model that utilizes dense combinations of dilated convolutions to obtain larger and more effective receptive fields. To better train this efficient generator, except for frequently-used VGG feature matching loss, we design a novel self-guided regression loss. We also employ a discriminator with local and global branches to ensure local-global contents consistency.
  arXiv  Detail & Related papers  (2020-02-07T03:45:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.