Towards Enhancing Fine-grained Details for Image Matting

• URL: http://arxiv.org/abs/2101.09095v1
• Date: Fri, 22 Jan 2021 13:20:23 GMT
• Title: Towards Enhancing Fine-grained Details for Image Matting
• Authors: Chang Liu, Henghui Ding, Xudong Jiang
• Abstract summary: We argue that recovering microscopic details relies on low-level but high-definition texture features. Our model consists of a conventional encoder-decoder Semantic Path and an independent down-sampling-free Textural Compensate Path. Our method outperforms previous start-of-the-art methods on the Composition-1k dataset.
• Score: 40.17208660790402
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, deep natural image matting has been rapidly evolved by extracting high-level contextual features into the model. However, most current methods still have difficulties with handling tiny details, like hairs or furs. In this paper, we argue that recovering these microscopic details relies on low-level but high-definition texture features. However, {these features are downsampled in a very early stage in current encoder-decoder-based models, resulting in the loss of microscopic details}. To address this issue, we design a deep image matting model {to enhance fine-grained details. Our model consists of} two parallel paths: a conventional encoder-decoder Semantic Path and an independent downsampling-free Textural Compensate Path (TCP). The TCP is proposed to extract fine-grained details such as lines and edges in the original image size, which greatly enhances the fineness of prediction. Meanwhile, to leverage the benefits of high-level context, we propose a feature fusion unit(FFU) to fuse multi-scale features from the semantic path and inject them into the TCP. In addition, we have observed that poorly annotated trimaps severely affect the performance of the model. Thus we further propose a novel term in loss function and a trimap generation method to improve our model's robustness to the trimaps. The experiments show that our method outperforms previous start-of-the-art methods on the Composition-1k dataset.

Related papers

• Distance Weighted Trans Network for Image Completion [52.318730994423106]
  We propose a new architecture that relies on Distance-based Weighted Transformer (DWT) to better understand the relationships between an image's components. CNNs are used to augment the local texture information of coarse priors. DWT blocks are used to recover certain coarse textures and coherent visual structures.
  arXiv  Detail & Related papers  (2023-10-11T12:46:11Z)
• Improving Pixel-based MIM by Reducing Wasted Modeling Capability [77.99468514275185]
  We propose a new method that explicitly utilizes low-level features from shallow layers to aid pixel reconstruction. To the best of our knowledge, we are the first to systematically investigate multi-level feature fusion for isotropic architectures. Our method yields significant performance gains, such as 1.2% on fine-tuning, 2.8% on linear probing, and 2.6% on semantic segmentation.
  arXiv  Detail & Related papers  (2023-08-01T03:44:56Z)
• Multi-Modality Deep Network for JPEG Artifacts Reduction [33.02405073842042]
  We propose a multimodal fusion learning method for text-guided JPEG artifacts reduction. Our method can obtain better deblocking results compared to the state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-04T11:54:02Z)
• Disentangled Pre-training for Image Matting [74.10407744483526]
  Image matting requires high-quality pixel-level human annotations to support the training of a deep model. We propose a self-supervised pre-training approach that can leverage infinite numbers of data to boost the matting performance.
  arXiv  Detail & Related papers  (2023-04-03T08:16:02Z)
• ARF: Artistic Radiance Fields [63.79314417413371]
  We present a method for transferring the artistic features of an arbitrary style image to a 3D scene. Previous methods that perform 3D stylization on point clouds or meshes are sensitive to geometric reconstruction errors. We propose to stylize the more robust radiance field representation.
  arXiv  Detail & Related papers  (2022-06-13T17:55:31Z)
• PP-Matting: High-Accuracy Natural Image Matting [11.68134059283327]
  PP-Matting is a trimap-free architecture that can achieve high-accuracy natural image matting. Our method applies a high-resolution detail branch (HRDB) that extracts fine-grained details of the foreground. Also, we propose a semantic context branch (SCB) that adopts a semantic segmentation subtask.
  arXiv  Detail & Related papers  (2022-04-20T12:54:06Z)
• Highly Efficient Natural Image Matting [15.977598189574659]
  We propose a trimap-free natural image matting method with a lightweight model. We construct an extremely light-weighted model, which achieves comparable performance with 1% (344k) of large models on popular natural image benchmarks.
  arXiv  Detail & Related papers  (2021-10-25T09:23:46Z)
• PixelPyramids: Exact Inference Models from Lossless Image Pyramids [58.949070311990916]
  Pixel-Pyramids is a block-autoregressive approach with scale-specific representations to encode the joint distribution of image pixels. It yields state-of-the-art results for density estimation on various image datasets, especially for high-resolution data. For CelebA-HQ 1024 x 1024, we observe that the density estimates are improved to 44% of the baseline despite sampling speeds superior even to easily parallelizable flow-based models.
  arXiv  Detail & Related papers  (2021-10-17T10:47:29Z)
• SDWNet: A Straight Dilated Network with Wavelet Transformation for Image Deblurring [23.86692375792203]
  Image deblurring is a computer vision problem that aims to recover a sharp image from a blurred image. Our model uses dilated convolution to enable the obtainment of the large receptive field with high spatial resolution. We propose a novel module using the wavelet transform, which effectively helps the network to recover clear high-frequency texture details.
  arXiv  Detail & Related papers  (2021-10-12T07:58:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.