PanORama: Multiview Consistent Panoptic Segmentation in Operating Rooms

• URL: http://arxiv.org/abs/2603.19920v1
• Date: Fri, 20 Mar 2026 13:04:07 GMT
• Title: PanORama: Multiview Consistent Panoptic Segmentation in Operating Rooms
• Abstract summary: We introduce PanORama, the first panoptic segmentation for the operating room that is multiview-consistent by design.<n>By modeling cross-view interactions at the feature level inside the backbone in a single forward pass, view consistency emerges directly rather than through post-hoc refinement.<n>PanORama is calibration-free, requiring no camera parameters, and generalizes to unseen camera viewpoints within any multiview configuration at inference time.
• Score: 39.219645333154794
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Operating rooms (ORs) are cluttered, dynamic, highly occluded environments, where reliable spatial understanding is essential for situational awareness during complex surgical workflows. Achieving spatial understanding for panoptic segmentation from sparse multiview images poses a fundamental challenge, as limited visibility in a subset of views often leads to mispredictions across cameras. To this end, we introduce PanORama, the first panoptic segmentation for the operating room that is multiview-consistent by design. By modeling cross-view interactions at the feature level inside the backbone in a single forward pass, view consistency emerges directly rather than through post-hoc refinement. We evaluate on the MM-OR and 4D-OR datasets, achieving >70% Panoptic Quality (PQ) performance, and outperforming the previous state of the art. Importantly, PanORama is calibration-free, requiring no camera parameters, and generalizes to unseen camera viewpoints within any multiview configuration at inference time. By substantially enhancing multiview segmentation and, consequently, spatial understanding in the OR, we believe our approach opens new opportunities for surgical perception and assistance. Code will be released upon acceptance.

Related papers

• Prisma-World: Camera-Controllable Multi-Agent Video World Model [67.72842238020192]
  We introduce Prisma-World, a camera-controllable multi-agent world model.<n>It formulates multi-agent generation as a joint geometry-aware denoising process for cross-view consistency.<n>Experiments show that a single Prisma-World model can generate high-fidelity multi-agent videos with flexible agent numbers.
  arXiv  Detail & Related papers  (2026-06-08T13:59:50Z)
• More than the Sum: Panorama-Language Models for Adverse Omni-Scenes [63.9800546079941]
  Existing vision-language models (VLMs) are tailored for pinhole imagery, stitching multiple narrow field-of-view inputs to piece together a complete omni-scene understanding.<n>We introduce the Panorama-Language Modeling (PLM)paradigm, a unified vision-language reasoning that is more than the sum of its pinhole counterparts.
  arXiv  Detail & Related papers  (2026-03-10T12:19:50Z)
• Panoramic Out-of-Distribution Segmentation for Autonomous Driving [25.32165408678185]
  We introduce Panoramic Out-of-distribution (PanOoS) to achieve comprehensive and safe scene understanding.<n> POS adapts to the characteristics of panoramic images through text-guided prompt distribution learning.<n> POS achieves leading closed-set segmentation capabilities and advances the development of panoramic understanding.
  arXiv  Detail & Related papers  (2025-05-06T13:51:26Z)
• Scene-Centric Unsupervised Panoptic Segmentation [56.253072379456356]
  Unsupervised panoptic segmentation aims to partition an image into semantically meaningful regions and distinct object instances without training on manually annotated data.<n>We present the first unsupervised panoptic method that directly trains on scene-centric imagery.
  arXiv  Detail & Related papers  (2025-04-02T17:58:46Z)
• MOSAIC: Generating Consistent, Privacy-Preserving Scenes from Multiple Depth Views in Multi-Room Environments [21.060917984023273]
  We introduce a diffusion-based approach for generating privacy-preserving digital twins of multi-room indoor environments from depth images only.<n>Central to our approach is a novel Multi-view Overlapped Scene Alignment with Implicit Consistency (MOSAIC) model.<n>Experiments show that MOSAIC outperforms state-of-the-art baselines on image fidelity metrics in reconstructing complex multi-room environments.
  arXiv  Detail & Related papers  (2025-03-18T01:50:57Z)
• Multi-view Video-Pose Pretraining for Operating Room Surgical Activity Recognition [5.787586057526269]
  Surgical activity recognition is a key computer vision task that detects activities or phases from multi-view camera recordings.<n>Existing SAR models often fail to account for fine-grained clinician movements and multi-view knowledge.<n>We propose a novel calibration-free multi-view multi-modal pretraining framework called Multiview Pretraining for Video-Pose Surgical Activity Recognition PreViPS.
  arXiv  Detail & Related papers  (2025-02-19T17:08:04Z)
• Panoramic Panoptic Segmentation: Insights Into Surrounding Parsing for Mobile Agents via Unsupervised Contrastive Learning [93.6645991946674]
  We introduce panoramic panoptic segmentation, as the most holistic scene understanding. A complete surrounding understanding provides a maximum of information to a mobile agent. We propose a framework which allows model training on standard pinhole images and transfers the learned features to a different domain.
  arXiv  Detail & Related papers  (2022-06-21T20:07:15Z)
• Panoramic Panoptic Segmentation: Towards Complete Surrounding Understanding via Unsupervised Contrastive Learning [97.37544023666833]
  We introduce panoramic panoptic segmentation as the most holistic scene understanding. A complete surrounding understanding provides a maximum of information to the agent. We propose a framework which allows model training on standard pinhole images and transfers the learned features to a different domain.
  arXiv  Detail & Related papers  (2021-03-01T09:37:27Z)
• Self-supervised Human Detection and Segmentation via Multi-view Consensus [116.92405645348185]
  We propose a multi-camera framework in which geometric constraints are embedded in the form of multi-view consistency during training. We show that our approach outperforms state-of-the-art self-supervised person detection and segmentation techniques on images that visually depart from those of standard benchmarks.
  arXiv  Detail & Related papers  (2020-12-09T15:47:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.