Occlusion-Aware Panoptic Segmentation with Joint Position Embedding and Occlusion-Level Attention

• URL: http://arxiv.org/abs/2607.18112v2
• Date: Tue, 21 Jul 2026 06:57:25 GMT
• Title: Occlusion-Aware Panoptic Segmentation with Joint Position Embedding and Occlusion-Level Attention
• Abstract summary: We propose Position Embedding Modulation with Occlusion Level Attention (PEMOLA)<n>PEMOLA can be seamlessly integrated into transformer-based panoptic segmentation.<n>Experiments on COCO-OLAC and Cityscapes-OLAC demonstrate that PEMOLA consistently improves panoptic segmentation quality.
• Score: 7.205486726857026
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Panoptic segmentation in complex scenes remains challenging because of occlusions, yet modern approaches often neglect occlusion modelling. In this paper, we propose Position Embedding Modulation with Occlusion Level Attention (PEMOLA), a novel occlusion-aware module that can be seamlessly integrated into transformer-based panoptic segmentation. To obtain occlusion cues, we train an occlusion classifier on the COCO-OLAC dataset. The classifier derives the occlusion-level attention, which serves as spatial guidance, while the occlusion labels are encoded into a learnable embedding to produce channel-wise weights. Through joint modulation, PEMOLA elegantly introduces the occlusion priors into the position embedding, thereby improving the occlusion modelling. We further annotate the Cityscapes dataset with occlusion levels, termed Cityscapes Occlusion Labels for All Computer Vision Tasks (Cityscapes-OLAC), following the same labelling protocol as COCO-OLAC, to evaluate the cross-dataset generalisation ability of PEMOLA. Extensive experiments on COCO-OLAC and Cityscapes-OLAC demonstrate that PEMOLA consistently improves panoptic segmentation quality while introducing minimal computational overhead. These results highlight the importance of occlusion modelling, where incorporating occlusion-level attention helps deliver robust panoptic segmentation under occlusion. Code and dataset are available at https://github.com/wenbo-wei/PEMOLA.

Related papers

• LIBERO-Occ: Evaluating and Improving Vision-Language-Action Models under Scene-Induced Occlusion via Viewpoint Imagination [66.06027569507403]
  textbfViewpoint Imagination (VIM) generates a complementary view from an occluded primary observation and conditions action prediction on both observed and imagined evidence.<n>VIM improves robustness across task suites, occlusion types, and severity levels without requiring additional cameras at deployment time.
  arXiv  Detail & Related papers  (2026-06-09T13:39:49Z)
• OcclusionFormer: Arranging Z-Order for Layout-Grounded Image Generation [49.49933361280724]
  We introduce OcclusionFormer, a novel framework that explicitly models Z-order priority by decoupling instances.<n>We also introduce a queried alignment loss that explicitly supervises individual instances and enhances semantic consistency.<n>The proposed method effectively reduces ambiguity in overlapping regions, enforces correct occlusion dependencies, and preserves structural integrity, leading to substantial accuracy gains across diverse scenes.
  arXiv  Detail & Related papers  (2026-05-20T16:10:44Z)
• Improved Object-Centric Diffusion Learning with Registers and Contrastive Alignment [83.56510119503265]
  Slot Attention (SA) with pretrained diffusion models has recently shown promise for object-centric learning (OCL), but suffers from slot entanglement and weak alignment between object slots and image content.<n>We propose Contrastive Object-centric Diffusion Alignment (CODA), a simple extension that (i) employs register slots to absorb residual attention and reduce interference between object slots, and (ii) applies a contrastive alignment loss to explicitly encourage slot-image correspondence.
  arXiv  Detail & Related papers  (2026-01-03T16:10:18Z)
• Harnessing Group-Oriented Consistency Constraints for Semi-Supervised Semantic Segmentation in CdZnTe Semiconductors [71.44213719783703]
  Intra-group Consistency Augmentation Framework (ICAF) developed to label Cadmium Zinc Telluride (CdZnTe) semiconductor images.<n>ICAF consists of two key modules, the View Augmentation Module (VAM) and the View Correction Module (VCM)<n>ICAF achieves a 70.6% mIoU on the CdZnTe dataset using only 2 group-annotated data.
  arXiv  Detail & Related papers  (2025-08-18T09:40:36Z)
• OccludeNet: A Causal Journey into Mixed-View Actor-Centric Video Action Recognition under Occlusions [37.79525665359017]
  We build OccludeNet, a large-scale occluded video dataset including both real and synthetic occlusion scenes.<n>Our analysis shows that actions with low scene relevance and partial body visibility see larger drops in accuracy.<n>We propose a structural causal model for occluded scenes and introduce the Causal Action Recognition (CAR) method, which uses backdoor adjustment and counterfactual reasoning.
  arXiv  Detail & Related papers  (2024-11-24T06:10:05Z)
• COCO-OLAC: A Benchmark for Occluded Panoptic Segmentation and Image Understanding [8.261771972240778]
  This paper proposes a new large-scale dataset named COCO-OLAC (COCO Occlusion Labels for All Computer Vision Tasks)<n> COCO-OLAC is derived from the COCO dataset by manually labelling images into three perceived occlusion levels.<n>We demonstrate that the proposed approach boosts the performance of the baseline model and achieves SOTA performance on the proposed COCO-OLAC dataset.
  arXiv  Detail & Related papers  (2024-09-19T13:26:28Z)
• Object-level Scene Deocclusion [92.39886029550286]
  We present a new self-supervised PArallel visible-to-COmplete diffusion framework, named PACO, for object-level scene deocclusion. To train PACO, we create a large-scale dataset with 500k samples to enable self-supervised learning. Experiments on COCOA and various real-world scenes demonstrate the superior capability of PACO for scene deocclusion, surpassing the state of the arts by a large margin.
  arXiv  Detail & Related papers  (2024-06-11T20:34:10Z)
• CMU-Flownet: Exploring Point Cloud Scene Flow Estimation in Occluded Scenario [10.852258389804984]
  Occlusions hinder point cloud frame alignment in LiDAR data, a challenge inadequately addressed by scene flow models. We introduce the Correlation Matrix Upsampling Flownet (CMU-Flownet), incorporating an occlusion estimation module within its cost volume layer. CMU-Flownet establishes state-of-the-art performance within the realms of occluded Flyingthings3D and KITTY datasets.
  arXiv  Detail & Related papers  (2024-04-16T13:47:21Z)
• OrCo: Towards Better Generalization via Orthogonality and Contrast for Few-Shot Class-Incremental Learning [57.43911113915546]
  Few-Shot Class-Incremental Learning (FSCIL) introduces a paradigm in which the problem space expands with limited data. FSCIL methods inherently face the challenge of catastrophic forgetting as data arrives incrementally. We propose the OrCo framework built on two core principles: features' orthogonality in the representation space, and contrastive learning.
  arXiv  Detail & Related papers  (2024-03-27T13:30:48Z)
• Peeking into occluded joints: A novel framework for crowd pose estimation [88.56203133287865]
  OPEC-Net is an Image-Guided Progressive GCN module that estimates invisible joints from an inference perspective. OCPose is the most complex Occluded Pose dataset with respect to average IoU between adjacent instances.
  arXiv  Detail & Related papers  (2020-03-23T19:32:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.