Inharmonious Region Localization by Magnifying Domain Discrepancy
- URL: http://arxiv.org/abs/2209.15368v1
- Date: Fri, 30 Sep 2022 10:41:16 GMT
- Title: Inharmonious Region Localization by Magnifying Domain Discrepancy
- Authors: Jing Liang, Li Niu, Penghao Wu, Fengjun Guo, Teng Long
- Abstract summary: Inharmonious region localization aims to localize the region in a synthetic image which is incompatible with surrounding background.
In this work, we tend to transform the input image to another color space to magnify the domain discrepancy between inharmonious region and background.
We present a novel framework consisting of a color mapping module and an inharmonious region localization network.
- Score: 18.661683923953085
- License: http://creativecommons.org/publicdomain/zero/1.0/
- Abstract: Inharmonious region localization aims to localize the region in a synthetic
image which is incompatible with surrounding background. The inharmony issue is
mainly attributed to the color and illumination inconsistency produced by image
editing techniques. In this work, we tend to transform the input image to
another color space to magnify the domain discrepancy between inharmonious
region and background, so that the model can identify the inharmonious region
more easily. To this end, we present a novel framework consisting of a color
mapping module and an inharmonious region localization network, in which the
former is equipped with a novel domain discrepancy magnification loss and the
latter could be an arbitrary localization network. Extensive experiments on
image harmonization dataset show the superiority of our designed framework. Our
code is available at
https://github.com/bcmi/MadisNet-Inharmonious-Region-Localization.
Related papers
- LIME: Localized Image Editing via Attention Regularization in Diffusion
Models [74.3811832586391]
This paper introduces LIME for localized image editing in diffusion models that do not require user-specified regions of interest (RoI) or additional text input.
Our method employs features from pre-trained methods and a simple clustering technique to obtain precise semantic segmentation maps.
We propose a novel cross-attention regularization technique that penalizes unrelated cross-attention scores in the RoI during the denoising steps, ensuring localized edits.
arXiv Detail & Related papers (2023-12-14T18:59:59Z) - Learning Global-aware Kernel for Image Harmonization [55.614444363743765]
Image harmonization aims to solve the visual inconsistency problem in composited images by adaptively adjusting the foreground pixels with the background as references.
Existing methods employ local color transformation or region matching between foreground and background, which neglects powerful proximity prior and independently distinguishes fore-/back-ground as a whole part for harmonization.
We propose a novel Global-aware Kernel Network (GKNet) to harmonize local regions with comprehensive consideration of long-distance background references.
arXiv Detail & Related papers (2023-05-19T13:49:02Z) - Towards Coherent Image Inpainting Using Denoising Diffusion Implicit
Models [43.83732051916894]
We propose COPAINT, which can coherently inpaint the whole image without introducing mismatches.
COPAINT also uses the Bayesian framework to jointly modify both revealed and unrevealed regions.
Our experiments verify that COPAINT can outperform the existing diffusion-based methods under both objective and subjective metrics.
arXiv Detail & Related papers (2023-04-06T18:35:13Z) - Region-Aware Diffusion for Zero-shot Text-driven Image Editing [78.58917623854079]
We propose a novel region-aware diffusion model (RDM) for entity-level image editing.
To strike a balance between image fidelity and inference speed, we design the intensive diffusion pipeline.
The results show that RDM outperforms the previous approaches in terms of visual quality, overall harmonization, non-editing region content preservation, and text-image semantic consistency.
arXiv Detail & Related papers (2023-02-23T06:20:29Z) - Inharmonious Region Localization with Auxiliary Style Feature [19.146209624835322]
Inharmonious region localization aims to localize the inharmonious region in a synthetic image.
We propose a novel color mapping module and a style feature loss to extract discriminative style features.
Based on the extracted style features, we also propose a novel style voting module to guide the localization of inharmonious region.
arXiv Detail & Related papers (2022-10-05T05:37:35Z) - MeshLoc: Mesh-Based Visual Localization [54.731309449883284]
We explore a more flexible alternative based on dense 3D meshes that does not require features matching between database images to build the scene representation.
Surprisingly competitive results can be obtained when extracting features on renderings of these meshes, without any neural rendering stage.
Our results show that dense 3D model-based representations are a promising alternative to existing representations and point to interesting and challenging directions for future research.
arXiv Detail & Related papers (2022-07-21T21:21:10Z) - Image Harmonization with Region-wise Contrastive Learning [51.309905690367835]
We propose a novel image harmonization framework with external style fusion and region-wise contrastive learning scheme.
Our method attempts to bring together corresponding positive and negative samples by maximizing the mutual information between the foreground and background styles.
arXiv Detail & Related papers (2022-05-27T15:46:55Z) - Image Harmonization by Matching Regional References [10.249228010611617]
Recent image harmonization methods typically summarize the appearance pattern of global background and apply it to the global foreground without location discrepancy.
For a real image, the appearances (illumination, color temperature, saturation, hue, texture, etc) of different regions can vary significantly.
Previous methods, which transfer the appearance globally, are not optimal.
arXiv Detail & Related papers (2022-04-10T16:23:06Z) - GLocal: Global Graph Reasoning and Local Structure Transfer for Person
Image Generation [2.580765958706854]
We focus on person image generation, namely, generating person image under various conditions, e.g., corrupted texture or different pose.
We present a GLocal framework to improve the occlusion-aware texture estimation by globally reasoning the style inter-correlations among different semantic regions.
For local structural information preservation, we further extract the local structure of the source image and regain it in the generated image via local structure transfer.
arXiv Detail & Related papers (2021-12-01T03:54:30Z) - Image Fine-grained Inpainting [89.17316318927621]
We present a one-stage model that utilizes dense combinations of dilated convolutions to obtain larger and more effective receptive fields.
To better train this efficient generator, except for frequently-used VGG feature matching loss, we design a novel self-guided regression loss.
We also employ a discriminator with local and global branches to ensure local-global contents consistency.
arXiv Detail & Related papers (2020-02-07T03:45:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.