ES-CRF: Embedded Superpixel CRF for Semantic Segmentation

• URL: http://arxiv.org/abs/2112.07106v1
• Date: Tue, 14 Dec 2021 02:06:28 GMT
• Title: ES-CRF: Embedded Superpixel CRF for Semantic Segmentation
• Authors: Jie Zhu, Huabin Huang, Banghuai Li, Leye Wang
• Abstract summary: We propose a novel method named Embedded Superpixel CRF (ES-CRF) to purify the feature representation of boundary pixels. ES-CRF fuses the CRF mechanism into the CNN network as an organic whole for more effective end-to-end optimization. It yields new records on two challenging benchmarks, i.e., Cityscapes and ADE20K.
• Score: 9.759391777814619
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Modern semantic segmentation methods devote much attention to adjusting feature representations to improve the segmentation performance in various ways, such as metric learning, architecture design, etc. However, almost all those methods neglect the particularity of boundary pixels. These pixels are prone to obtain confusing features from both sides due to the continuous expansion of receptive fields in CNN networks. In this way, they will mislead the model optimization direction and make the class weights of such categories that tend to share many adjacent pixels lack discrimination, which will damage the overall performance. In this work, we dive deep into this problem and propose a novel method named Embedded Superpixel CRF (ES-CRF) to address it. ES-CRF involves two main aspects. On the one hand, ES-CRF innovatively fuses the CRF mechanism into the CNN network as an organic whole for more effective end-to-end optimization. It utilizes CRF to guide the message passing between pixels in high-level features to purify the feature representation of boundary pixels, with the help of inner pixels belong to the same object. On the other hand, superpixel is integrated into ES-CRF to exploit the local object prior for more reliable message passing. Finally, our proposed method yields new records on two challenging benchmarks, i.e., Cityscapes and ADE20K. Moreover, we make detailed theoretical analysis to verify the superiority of ES-CRF.

Related papers

• High-resolution Photo Enhancement in Real-time: A Laplacian Pyramid Network [73.19214585791268]
  This paper introduces a pyramid network called LLF-LUT++, which integrates global and local operators through closed-form Laplacian pyramid decomposition and reconstruction.<n>Specifically, we utilize an image-adaptive 3D LUT that capitalizes on the global tonal characteristics of downsampled images.<n>LLF-LUT++ not only achieves a 2.64 dB improvement in PSNR on the HDR+ dataset, but also further reduces, with 4K resolution images processed in just 13 ms on a single GPU.
  arXiv  Detail & Related papers  (2025-10-13T16:52:32Z)
• Leveraging Adaptive Implicit Representation Mapping for Ultra High-Resolution Image Segmentation [19.87987918759425]
  Implicit representation mapping (IRM) can translate image features to any continuous resolution, showcasing its potent capability for ultra-high-resolution image segmentation refinement. Current IRM-based methods for refining ultra-high-resolution image segmentation often rely on CNN-based encoders to extract image features. We propose a novel approach that leverages the newly proposed Implicit Representation Mapping (AIRM) for ultra-high-resolution Image Function.
  arXiv  Detail & Related papers  (2024-07-31T00:34:37Z)
• LeRF: Learning Resampling Function for Adaptive and Efficient Image Interpolation [64.34935748707673]
  Recent deep neural networks (DNNs) have made impressive progress in performance by introducing learned data priors. We propose a novel method of Learning Resampling (termed LeRF) which takes advantage of both the structural priors learned by DNNs and the locally continuous assumption. LeRF assigns spatially varying resampling functions to input image pixels and learns to predict the shapes of these resampling functions with a neural network.
  arXiv  Detail & Related papers  (2024-07-13T16:09:45Z)
• SANeRF-HQ: Segment Anything for NeRF in High Quality [61.77762568224097]
  We introduce the Segment Anything for NeRF in High Quality (SANeRF-HQ) to achieve high-quality 3D segmentation of any target object in a given scene. We employ density field and RGB similarity to enhance the accuracy of segmentation boundary during the aggregation.
  arXiv  Detail & Related papers  (2023-12-03T23:09:38Z)
• GP-NeRF: Generalized Perception NeRF for Context-Aware 3D Scene Understanding [101.32590239809113]
  Generalized Perception NeRF (GP-NeRF) is a novel pipeline that makes the widely used segmentation model and NeRF work compatibly under a unified framework. We propose two self-distillation mechanisms, i.e., the Semantic Distill Loss and the Depth-Guided Semantic Distill Loss, to enhance the discrimination and quality of the semantic field.
  arXiv  Detail & Related papers  (2023-11-20T15:59:41Z)
• Efficient fine-grained road segmentation using superpixel-based CNN and CRF models [0.0]
  We propose a novel approach to utilise the advantages of CNNs for the task of road segmentation at reasonable computational effort. The proposed system obtained comparable performance among the top performing algorithms on the KITTI road benchmark.
  arXiv  Detail & Related papers  (2022-06-22T12:38:30Z)
• Rethinking Unsupervised Neural Superpixel Segmentation [6.123324869194195]
  unsupervised learning for superpixel segmentation via CNNs has been studied. We propose three key elements to improve the efficacy of such networks. By experimenting with the BSDS500 dataset, we find evidence to the significance of our proposal.
  arXiv  Detail & Related papers  (2022-06-21T09:30:26Z)
• NeW CRFs: Neural Window Fully-connected CRFs for Monocular Depth Estimation [42.062788492398674]
  Estimating the accurate depth from a single image is challenging since it is inherently ambiguous and ill-posed. We take the path of CRFs optimization and leverage the potential of fully-connected CRFs. Our method significantly improves the performance across all metrics on both the KITTI and NYUv2 datasets.
  arXiv  Detail & Related papers  (2022-03-03T03:27:20Z)
• Semantic Segmentation by Improved Generative Adversarial Networks [0.0]
  We introduce Convolutional CRFs (ConvCRFs) as an effective improvement solution for the image semantic segmentation task. Our method not only learns an end-to-end mapping from input image to corresponding output image, but also learns a loss function to train this mapping.
  arXiv  Detail & Related papers  (2021-04-20T11:59:29Z)
• Asymmetric CNN for image super-resolution [102.96131810686231]
  Deep convolutional neural networks (CNNs) have been widely applied for low-level vision over the past five years. We propose an asymmetric CNN (ACNet) comprising an asymmetric block (AB), a mem?ory enhancement block (MEB) and a high-frequency feature enhancement block (HFFEB) for image super-resolution. Our ACNet can effectively address single image super-resolution (SISR), blind SISR and blind SISR of blind noise problems.
  arXiv  Detail & Related papers  (2021-03-25T07:10:46Z)
• AINet: Association Implantation for Superpixel Segmentation [82.21559299694555]
  We propose a novel textbfAssociation textbfImplantation (AI) module to enable the network to explicitly capture the relations between the pixel and its surrounding grids. Our method could not only achieve state-of-the-art performance but maintain satisfactory inference efficiency.
  arXiv  Detail & Related papers  (2021-01-26T10:40:13Z)
• CARAFE++: Unified Content-Aware ReAssembly of FEatures [132.49582482421246]
  We propose unified Content-Aware ReAssembly of FEatures (CARAFE++), a universal, lightweight and highly effective operator to fulfill this goal. CARAFE++ generates adaptive kernels on-the-fly to enable instance-specific content-aware handling. It shows consistent and substantial gains across all the tasks with negligible computational overhead.
  arXiv  Detail & Related papers  (2020-12-07T07:34:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.