Panoptic-Depth Color Map for Combination of Depth and Image Segmentation

• URL: http://arxiv.org/abs/2308.12937v1
• Date: Thu, 24 Aug 2023 17:25:09 GMT
• Title: Panoptic-Depth Color Map for Combination of Depth and Image Segmentation
• Authors: Jia-Quan Yu, Soo-Chang Pei
• Abstract summary: We propose an innovative approach to combine image segmentation and depth estimation. By incorporating an additional depth estimation branch into the segmentation network, it can predict the depth of each instance segment. Our proposed method demonstrates a new possibility of combining different tasks and networks to generate a more comprehensive image recognition result.
• Score: 2.33877878310217
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Image segmentation and depth estimation are crucial tasks in computer vision, especially in autonomous driving scenarios. Although these tasks are typically addressed separately, we propose an innovative approach to combine them in our novel deep learning network, Panoptic-DepthLab. By incorporating an additional depth estimation branch into the segmentation network, it can predict the depth of each instance segment. Evaluating on Cityscape dataset, we demonstrate the effectiveness of our method in achieving high-quality segmentation results with depth and visualize it with a color map. Our proposed method demonstrates a new possibility of combining different tasks and networks to generate a more comprehensive image recognition result to facilitate the safety of autonomous driving vehicles.

Related papers

• Depth-guided Texture Diffusion for Image Semantic Segmentation [47.46257473475867]
  We introduce a Depth-guided Texture Diffusion approach that effectively tackles the outlined challenge. Our method extracts low-level features from edges and textures to create a texture image. By integrating this enriched depth map with the original RGB image into a joint feature embedding, our method effectively bridges the disparity between the depth map and the image.
  arXiv  Detail & Related papers  (2024-08-17T04:55:03Z)
• Towards Deeply Unified Depth-aware Panoptic Segmentation with Bi-directional Guidance Learning [63.63516124646916]
  We propose a deeply unified framework for depth-aware panoptic segmentation. We propose a bi-directional guidance learning approach to facilitate cross-task feature learning. Our method sets the new state of the art for depth-aware panoptic segmentation on both Cityscapes-DVPS and SemKITTI-DVPS datasets.
  arXiv  Detail & Related papers  (2023-07-27T11:28:33Z)
• SurroundDepth: Entangling Surrounding Views for Self-Supervised Multi-Camera Depth Estimation [101.55622133406446]
  We propose a SurroundDepth method to incorporate the information from multiple surrounding views to predict depth maps across cameras. Specifically, we employ a joint network to process all the surrounding views and propose a cross-view transformer to effectively fuse the information from multiple views. In experiments, our method achieves the state-of-the-art performance on the challenging multi-camera depth estimation datasets.
  arXiv  Detail & Related papers  (2022-04-07T17:58:47Z)
• X-Distill: Improving Self-Supervised Monocular Depth via Cross-Task Distillation [69.9604394044652]
  We propose a novel method to improve the self-supervised training of monocular depth via cross-task knowledge distillation. During training, we utilize a pretrained semantic segmentation teacher network and transfer its semantic knowledge to the depth network. We extensively evaluate the efficacy of our proposed approach on the KITTI benchmark and compare it with the latest state of the art.
  arXiv  Detail & Related papers  (2021-10-24T19:47:14Z)
• Self-Supervised Monocular Depth Estimation with Internal Feature Fusion [12.874712571149725]
  Self-supervised learning for depth estimation uses geometry in image sequences for supervision. We propose a novel depth estimation networkDIFFNet, which can make use of semantic information in down and upsampling procedures.
  arXiv  Detail & Related papers  (2021-10-18T17:31:11Z)
• Sparse Auxiliary Networks for Unified Monocular Depth Prediction and Completion [56.85837052421469]
  Estimating scene geometry from data obtained with cost-effective sensors is key for robots and self-driving cars. In this paper, we study the problem of predicting dense depth from a single RGB image with optional sparse measurements from low-cost active depth sensors. We introduce Sparse Networks (SANs), a new module enabling monodepth networks to perform both the tasks of depth prediction and completion.
  arXiv  Detail & Related papers  (2021-03-30T21:22:26Z)
• SOSD-Net: Joint Semantic Object Segmentation and Depth Estimation from Monocular images [94.36401543589523]
  We introduce the concept of semantic objectness to exploit the geometric relationship of these two tasks. We then propose a Semantic Object and Depth Estimation Network (SOSD-Net) based on the objectness assumption. To the best of our knowledge, SOSD-Net is the first network that exploits the geometry constraint for simultaneous monocular depth estimation and semantic segmentation.
  arXiv  Detail & Related papers  (2021-01-19T02:41:03Z)
• Semantic-Guided Representation Enhancement for Self-supervised Monocular Trained Depth Estimation [39.845944724079814]
  Self-supervised depth estimation has shown its great effectiveness in producing high quality depth maps given only image sequences as input. However, its performance usually drops when estimating on border areas or objects with thin structures due to the limited depth representation ability. We propose a semantic-guided depth representation enhancement method, which promotes both local and global depth feature representations.
  arXiv  Detail & Related papers  (2020-12-15T02:24:57Z)
• The Edge of Depth: Explicit Constraints between Segmentation and Depth [25.232436455640716]
  We study the mutual benefits of two common computer vision tasks, self-supervised depth estimation and semantic segmentation from images. We propose to explicitly measure the border consistency between segmentation and depth and minimize it. Through extensive experiments, our proposed approach advances the state of the art on unsupervised monocular depth estimation in the KITTI.
  arXiv  Detail & Related papers  (2020-04-01T00:03:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.