Video Semantic Segmentation with Distortion-Aware Feature Correction

• URL: http://arxiv.org/abs/2006.10380v2
• Date: Thu, 12 Nov 2020 09:12:50 GMT
• Title: Video Semantic Segmentation with Distortion-Aware Feature Correction
• Authors: Jiafan Zhuang, Zilei Wang, Bingke Wang
• Abstract summary: Per-frame image segmentation is generally unacceptable in practice due to high computation cost. We propose distortion-aware feature correction to alleviate the issue. Our proposed method can significantly boost the accuracy of video semantic segmentation at a low price.
• Score: 32.00672651803015
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video semantic segmentation is active in recent years benefited from the great progress of image semantic segmentation. For such a task, the per-frame image segmentation is generally unacceptable in practice due to high computation cost. To tackle this issue, many works use the flow-based feature propagation to reuse the features of previous frames. However, the optical flow estimation inevitably suffers inaccuracy and then causes the propagated features distorted. In this paper, we propose distortion-aware feature correction to alleviate the issue, which improves video segmentation performance by correcting distorted propagated features. To be specific, we firstly propose to transfer distortion patterns from feature into image space and conduct effective distortion map prediction. Benefited from the guidance of distortion maps, we proposed Feature Correction Module (FCM) to rectify propagated features in the distorted areas. Our proposed method can significantly boost the accuracy of video semantic segmentation at a low price. The extensive experimental results on Cityscapes and CamVid show that our method outperforms the recent state-of-the-art methods.

Related papers

• Transparency Distortion Robustness for SOTA Image Segmentation Tasks [4.1119273264193685]
  We propose a method to synthetically augment existing datasets with spatially varying distortions. Our experiments show, that these distortion effects degrade the performance of state-of-the-art segmentation models.
  arXiv  Detail & Related papers  (2024-05-21T15:30:25Z)
• DGNet: Dynamic Gradient-Guided Network for Water-Related Optics Image Enhancement [77.0360085530701]
  Underwater image enhancement (UIE) is a challenging task due to the complex degradation caused by underwater environments. Previous methods often idealize the degradation process, and neglect the impact of medium noise and object motion on the distribution of image features. Our approach utilizes predicted images to dynamically update pseudo-labels, adding a dynamic gradient to optimize the network's gradient space.
  arXiv  Detail & Related papers  (2023-12-12T06:07:21Z)
• Corner-to-Center Long-range Context Model for Efficient Learned Image Compression [70.0411436929495]
  In the framework of learned image compression, the context model plays a pivotal role in capturing the dependencies among latent representations. We propose the textbfCorner-to-Center transformer-based Context Model (C$3$M) designed to enhance context and latent predictions. In addition, to enlarge the receptive field in the analysis and synthesis transformation, we use the Long-range Crossing Attention Module (LCAM) in the encoder/decoder.
  arXiv  Detail & Related papers  (2023-11-29T21:40:28Z)
• Fast Hybrid Image Retargeting [0.0]
  We propose a method that quantifies and limits warping distortions with the use of content-aware cropping. Our method outperforms recent approaches, while running in a fraction of their execution time.
  arXiv  Detail & Related papers  (2022-03-25T11:46:06Z)
• Correcting Face Distortion in Wide-Angle Videos [85.88898349347149]
  We present a video warping algorithm to correct these distortions. Our key idea is to apply stereographic projection locally on the facial regions. For performance evaluation, we develop a wide-angle video dataset with a wide range of focal lengths.
  arXiv  Detail & Related papers  (2021-11-18T21:28:17Z)
• Wide-angle Image Rectification: A Survey [86.36118799330802]
  wide-angle images contain distortions that violate the assumptions underlying pinhole camera models. Image rectification, which aims to correct these distortions, can solve these problems. We present a detailed description and discussion of the camera models used in different approaches. Next, we review both traditional geometry-based image rectification methods and deep learning-based methods.
  arXiv  Detail & Related papers  (2020-10-30T17:28:40Z)
• Adaptive Debanding Filter [55.42929350861115]
  Banding artifacts manifest as staircase-like color bands on pictures or video frames. We propose a content-adaptive smoothing filtering followed by dithered quantization, as a post-processing module. Experimental results show that our proposed debanding filter outperforms state-of-the-art false contour removing algorithms both visually and quantitatively.
  arXiv  Detail & Related papers  (2020-09-22T20:44:20Z)
• A Deep Ordinal Distortion Estimation Approach for Distortion Rectification [62.72089758481803]
  We propose a novel distortion rectification approach that can obtain more accurate parameters with higher efficiency. We design a local-global associated estimation network that learns the ordinal distortion to approximate the realistic distortion distribution. Considering the redundancy of distortion information, our approach only uses a part of distorted image for the ordinal distortion estimation.
  arXiv  Detail & Related papers  (2020-07-21T10:03:42Z)
• UnRectDepthNet: Self-Supervised Monocular Depth Estimation using a Generic Framework for Handling Common Camera Distortion Models [8.484676769284578]
  We propose a generic scale-aware self-supervised pipeline for estimating depth, euclidean distance, and visual odometry from unrectified monocular videos. The proposed algorithm is evaluated further on the KITTI rectified dataset, and we achieve state-of-the-art results.
  arXiv  Detail & Related papers  (2020-07-13T20:35:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.