Real-time Semantic Segmentation with Context Aggregation Network

• URL: http://arxiv.org/abs/2011.00993v2
• Date: Sun, 11 Apr 2021 20:51:58 GMT
• Title: Real-time Semantic Segmentation with Context Aggregation Network
• Authors: Michael Ying Yang, Saumya Kumaar, Ye Lyu, Francesco Nex
• Abstract summary: We propose a dual branch convolutional neural network, with significantly lower computational costs as compared to the state-of-the-art. We evaluate our method on two semantic segmentation datasets, namely Cityscapes dataset and UAVid dataset.
• Score: 14.560708848716754
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the increasing demand of autonomous systems, pixelwise semantic segmentation for visual scene understanding needs to be not only accurate but also efficient for potential real-time applications. In this paper, we propose Context Aggregation Network, a dual branch convolutional neural network, with significantly lower computational costs as compared to the state-of-the-art, while maintaining a competitive prediction accuracy. Building upon the existing dual branch architectures for high-speed semantic segmentation, we design a cheap high resolution branch for effective spatial detailing and a context branch with light-weight versions of global aggregation and local distribution blocks, potent to capture both long-range and local contextual dependencies required for accurate semantic segmentation, with low computational overheads. We evaluate our method on two semantic segmentation datasets, namely Cityscapes dataset and UAVid dataset. For Cityscapes test set, our model achieves state-of-the-art results with mIOU of 75.9%, at 76 FPS on an NVIDIA RTX 2080Ti and 8 FPS on a Jetson Xavier NX. With regards to UAVid dataset, our proposed network achieves mIOU score of 63.5% with high execution speed (15 FPS).

Related papers

• PIDNet: A Real-time Semantic Segmentation Network Inspired by PID Controllers [6.0653144230649865]
  Two-branch network architecture has shown its efficiency and effectiveness in real-time semantic segmentation tasks. We propose a novel three-branch network architecture: PIDNet, which contains three branches to parse detailed, context and boundary information. Our family of PIDNets achieve the best trade-off between inference speed and accuracy and their accuracy surpasses all the existing models with similar inference speed on the Cityscapes and CamVid datasets.
  arXiv  Detail & Related papers  (2022-06-04T23:16:52Z)
• FBSNet: A Fast Bilateral Symmetrical Network for Real-Time Semantic Segmentation [23.25851281719734]
  We propose a Fast Bilateral Symmetrical Network (FBSNet) for real-time semantic segmentation. FBSNet employs a symmetrical-decoder structure with two branches, semantic information branch, and spatial detail branch. The experimental results of Cityscapes and CamVid show that the proposed FBSNet can strike a good balance between accuracy and efficiency.
  arXiv  Detail & Related papers  (2021-09-02T04:16:39Z)
• Real-time Semantic Segmentation with Fast Attention [94.88466483540692]
  We propose a novel architecture for semantic segmentation of high-resolution images and videos in real-time. The proposed architecture relies on our fast spatial attention, which is a simple yet efficient modification of the popular self-attention mechanism. We show that results on multiple datasets demonstrate superior performance with better accuracy and speed compared to existing approaches.
  arXiv  Detail & Related papers  (2020-07-07T22:37:16Z)
• Real-time Semantic Segmentation via Spatial-detail Guided Context Propagation [49.70144583431999]
  We propose the spatial-detail guided context propagation network (SGCPNet) for achieving real-time semantic segmentation. It uses the spatial details of shallow layers to guide the propagation of the low-resolution global contexts, in which the lost spatial information can be effectively reconstructed. It achieves 69.5% mIoU segmentation accuracy, while its speed reaches 178.5 FPS on 768x1536 images on a GeForce GTX 1080 Ti GPU card.
  arXiv  Detail & Related papers  (2020-05-22T07:07:26Z)
• BiSeNet V2: Bilateral Network with Guided Aggregation for Real-time Semantic Segmentation [118.46210049742993]
  We propose an efficient and effective architecture with a good trade-off between speed and accuracy, termed Bilateral spatial Network (BiSeNet V2) For a 2,048x1, input, we achieve 72.6% Mean IoU on the Cityscapes test set with a speed of 156 FPS on one NVIDIA GeForce 1080 Ti card, which is significantly faster than existing methods, yet we achieve better segmentation accuracy.
  arXiv  Detail & Related papers  (2020-04-05T10:26:38Z)
• Real-Time Semantic Segmentation via Auto Depth, Downsampling Joint Decision and Feature Aggregation [54.28963233377946]
  We propose a joint search framework, called AutoRTNet, to automate the design of segmentation strategies. Specifically, we propose hyper-cells to jointly decide the network depth and downsampling strategy, and an aggregation cell to achieve automatic multi-scale feature aggregation.
  arXiv  Detail & Related papers  (2020-03-31T14:02:25Z)
• Real-Time High-Performance Semantic Image Segmentation of Urban Street Scenes [98.65457534223539]
  We propose a real-time high-performance DCNN-based method for robust semantic segmentation of urban street scenes. The proposed method achieves the accuracy of 73.6% and 68.0% mean Intersection over Union (mIoU) with the inference speed of 51.0 fps and 39.3 fps.
  arXiv  Detail & Related papers  (2020-03-11T08:45:53Z)
• Efficient Video Semantic Segmentation with Labels Propagation and Refinement [138.55845680523908]
  This paper tackles the problem of real-time semantic segmentation of high definition videos using a hybrid GPU / CPU approach. We propose an Efficient Video(EVS) pipeline that combines: (i) On the CPU, a very fast optical flow method, that is used to exploit the temporal aspect of the video and propagate semantic information from one frame to the next. On the popular Cityscapes dataset with high resolution frames (2048 x 1024), the proposed operating points range from 80 to 1000 Hz on a single GPU and CPU.
  arXiv  Detail & Related papers  (2019-12-26T11:45:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.