Part-aware Panoptic Segmentation
- URL: http://arxiv.org/abs/2106.06351v1
- Date: Fri, 11 Jun 2021 12:48:07 GMT
- Title: Part-aware Panoptic Segmentation
- Authors: Daan de Geus, Panagiotis Meletis, Chenyang Lu, Xiaoxiao Wen, Gijs
Dubbelman
- Abstract summary: Part-aware Panoptic (PPS) aims to understand a scene at multiple levels of abstraction.
We provide consistent annotations on two commonly used datasets: Cityscapes and Pascal VOC.
We present a single metric to evaluate PPS, called Part-aware Panoptic Quality (PartPQ)
- Score: 3.342126234995932
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In this work, we introduce the new scene understanding task of Part-aware
Panoptic Segmentation (PPS), which aims to understand a scene at multiple
levels of abstraction, and unifies the tasks of scene parsing and part parsing.
For this novel task, we provide consistent annotations on two commonly used
datasets: Cityscapes and Pascal VOC. Moreover, we present a single metric to
evaluate PPS, called Part-aware Panoptic Quality (PartPQ). For this new task,
using the metric and annotations, we set multiple baselines by merging results
of existing state-of-the-art methods for panoptic segmentation and part
segmentation. Finally, we conduct several experiments that evaluate the
importance of the different levels of abstraction in this single task.
Related papers
- Task-aligned Part-aware Panoptic Segmentation through Joint Object-Part Representations [2.087148326341881]
Part-aware panoptic segmentation (PPS) requires (a) that each foreground object and background region in an image is segmented and classified, and (b) that all parts within foreground objects are segmented, classified and linked to their parent object.
Existing methods approach PPS by separately conducting object-level and part-level segmentation.
We propose Task-aware Part-Aligned Panoptic (TAPPS)
TAPPS learns to predict part-level segments that are linked to individual parent objects, aligning the learning objective with the task objective, and allowing TAPPS to leverage joint object-part representations.
arXiv Detail & Related papers (2024-06-14T15:20:46Z) - JPPF: Multi-task Fusion for Consistent Panoptic-Part Segmentation [12.19926973291957]
Part-aware panoptic segmentation is a problem of computer vision that aims to provide a semantic understanding of the scene at multiple levels of granularity.
We present our Joint Panoptic Part Fusion (JPPF) that combines the three individual segmentations effectively to obtain a panoptic-part segmentation.
arXiv Detail & Related papers (2023-11-30T15:17:46Z) - PartSeg: Few-shot Part Segmentation via Part-aware Prompt Learning [44.48704588318053]
We develop a novel method termed PartSeg for few-shot part segmentation based on multimodal learning.
We conduct extensive experiments on the PartImageNet and Pascal$_$Part datasets.
arXiv Detail & Related papers (2023-08-24T13:03:42Z) - Towards Deeply Unified Depth-aware Panoptic Segmentation with
Bi-directional Guidance Learning [63.63516124646916]
We propose a deeply unified framework for depth-aware panoptic segmentation.
We propose a bi-directional guidance learning approach to facilitate cross-task feature learning.
Our method sets the new state of the art for depth-aware panoptic segmentation on both Cityscapes-DVPS and SemKITTI-DVPS datasets.
arXiv Detail & Related papers (2023-07-27T11:28:33Z) - AIMS: All-Inclusive Multi-Level Segmentation [93.5041381700744]
We propose a new task, All-Inclusive Multi-Level (AIMS), which segments visual regions into three levels: part, entity, and relation.
We also build a unified AIMS model through multi-dataset multi-task training to address the two major challenges of annotation inconsistency and task correlation.
arXiv Detail & Related papers (2023-05-28T16:28:49Z) - GaitStrip: Gait Recognition via Effective Strip-based Feature
Representations and Multi-Level Framework [34.397404430838286]
We present a strip-based multi-level gait recognition network, named GaitStrip, to extract comprehensive gait information at different levels.
To be specific, our high-level branch explores the context of gait sequences and our low-level one focuses on detailed posture changes.
Our GaitStrip achieves state-of-the-art performance in both normal walking and complex conditions.
arXiv Detail & Related papers (2022-03-08T09:49:48Z) - STEP: Segmenting and Tracking Every Pixel [107.23184053133636]
We present a new benchmark: Segmenting and Tracking Every Pixel (STEP)
Our work is the first that targets this task in a real-world setting that requires dense interpretation in both spatial and temporal domains.
For measuring the performance, we propose a novel evaluation metric and Tracking Quality (STQ)
arXiv Detail & Related papers (2021-02-23T18:43:02Z) - Video Panoptic Segmentation [117.08520543864054]
We propose and explore a new video extension of this task, called video panoptic segmentation.
To invigorate research on this new task, we present two types of video panoptic datasets.
We propose a novel video panoptic segmentation network (VPSNet) which jointly predicts object classes, bounding boxes, masks, instance id tracking, and semantic segmentation in video frames.
arXiv Detail & Related papers (2020-06-19T19:35:47Z) - Motion-supervised Co-Part Segmentation [88.40393225577088]
We propose a self-supervised deep learning method for co-part segmentation.
Our approach develops the idea that motion information inferred from videos can be leveraged to discover meaningful object parts.
arXiv Detail & Related papers (2020-04-07T09:56:45Z) - SceneEncoder: Scene-Aware Semantic Segmentation of Point Clouds with A
Learnable Scene Descriptor [51.298760338410624]
We propose a SceneEncoder module to impose a scene-aware guidance to enhance the effect of global information.
The module predicts a scene descriptor, which learns to represent the categories of objects existing in the scene.
We also design a region similarity loss to propagate distinguishing features to their own neighboring points with the same label.
arXiv Detail & Related papers (2020-01-24T16:53:30Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.