SAP: Segment Any 4K Panorama

• URL: http://arxiv.org/abs/2603.12759v1
• Date: Fri, 13 Mar 2026 08:02:38 GMT
• Title: SAP: Segment Any 4K Panorama
• Abstract summary: We introduce Segment Any 4K Panorama (SAP), a foundation model for 4K high-resolution panoramic instance-level segmentation.<n>We reformulate panoramic segmentation as fixed-trajectory perspective video segmentation, decomposing a panorama into overlapping perspective patches sampled along a continuous spherical.<n>This memory-aligned reformulation preserves native 4K resolution while restoring the smooth viewpoint transitions required for stable cross-view propagation.
• Score: 64.69300263307441
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Promptable instance segmentation is widely adopted in embodied and AR systems, yet the performance of foundation models trained on perspective imagery often degrades on 360° panoramas. In this paper, we introduce Segment Any 4K Panorama (SAP), a foundation model for 4K high-resolution panoramic instance-level segmentation. We reformulate panoramic segmentation as fixed-trajectory perspective video segmentation, decomposing a panorama into overlapping perspective patches sampled along a continuous spherical traversal. This memory-aligned reformulation preserves native 4K resolution while restoring the smooth viewpoint transitions required for stable cross-view propagation. To enable large-scale supervision, we synthesize 183,440 4K-resolution panoramic images with instance segmentation labels using the InfiniGen engine. Trained under this trajectory-aligned paradigm, SAP generalizes effectively to real-world 360° images, achieving +17.2 zero-shot mIoU gain over vanilla SAM2 of different sizes on real-world 4K panorama benchmark.

Related papers

• Pano2World: End-to-End 3D Generation via Unified Multi-View Sequences [9.89397063309918]
  We present Pano2World, which takes a single indoor panorama as input and directly outputs a persistent, explorable 3D Gaussian scene.<n> Experiments demonstrate that Pano2World significantly outperforms existing methods on the multi-position panoramic novel-view synthesis benchmark.
  arXiv  Detail & Related papers  (2026-07-01T11:54:02Z)
• TiP4GEN: Text to Immersive Panorama 4D Scene Generation [82.8444414014506]
  TiP4GEN is a text-to-dynamic panorama scene generation framework.<n>It enables fine-grained content control and synthesizes motion-rich, geometry-consistent panoramic 4D scenes.<n> TiP4GEN integrates panorama video generation and dynamic scene reconstruction to create 360-degree immersive virtual environments.
  arXiv  Detail & Related papers  (2025-08-17T16:02:24Z)
• Panoramic Out-of-Distribution Segmentation for Autonomous Driving [25.32165408678185]
  We introduce Panoramic Out-of-distribution (PanOoS) to achieve comprehensive and safe scene understanding.<n> POS adapts to the characteristics of panoramic images through text-guided prompt distribution learning.<n> POS achieves leading closed-set segmentation capabilities and advances the development of panoramic understanding.
  arXiv  Detail & Related papers  (2025-05-06T13:51:26Z)
• HoloTime: Taming Video Diffusion Models for Panoramic 4D Scene Generation [29.579493980120173]
  HoloTime is a framework that integrates video diffusion models to generate panoramic videos from a single prompt or reference image.<n>360World dataset is the first comprehensive collection of panoramic videos suitable for downstream 4D scene reconstruction tasks.<n>Panoramic Animator is a two-stage image-to-video diffusion model that can convert panoramic images into high-quality panoramic videos.<n>Panoramic Space-Time Reconstruction uses a space-time depth estimation method to transform the generated panoramic videos into 4D point clouds.
  arXiv  Detail & Related papers  (2025-04-30T13:55:28Z)
• DiffPano: Scalable and Consistent Text to Panorama Generation with Spherical Epipolar-Aware Diffusion [60.45000652592418]
  We propose a novel text-driven panoramic generation framework, DiffPano, to achieve scalable, consistent, and diverse panoramic scene generation. We show that DiffPano can generate consistent, diverse panoramic images with given unseen text descriptions and camera poses.
  arXiv  Detail & Related papers  (2024-10-31T17:57:02Z)
• 4K4DGen: Panoramic 4D Generation at 4K Resolution [67.98105958108503]
  We tackle the challenging task of elevating a single panorama to an immersive 4D experience. For the first time, we demonstrate the capability to generate omnidirectional dynamic scenes with 360$circ$ views at 4K resolution. We achieve high-quality Panorama-to-4D generation at a resolution of 4K for the first time.
  arXiv  Detail & Related papers  (2024-06-19T13:11:02Z)
• ClusterFormer: Clustering As A Universal Visual Learner [80.79669078819562]
  CLUSTERFORMER is a universal vision model based on the CLUSTERing paradigm with TransFORMER. It is capable of tackling heterogeneous vision tasks with varying levels of clustering granularity. For its efficacy, we hope our work can catalyze a paradigm shift in universal models in computer vision.
  arXiv  Detail & Related papers  (2023-09-22T22:12:30Z)
• PanoGRF: Generalizable Spherical Radiance Fields for Wide-baseline Panoramas [54.4948540627471]
  We propose PanoGRF, Generalizable Spherical Radiance Fields for Wide-baseline Panoramas. Unlike generalizable radiance fields trained on perspective images, PanoGRF avoids the information loss from panorama-to-perspective conversion. Results on multiple panoramic datasets demonstrate that PanoGRF significantly outperforms state-of-the-art generalizable view synthesis methods.
  arXiv  Detail & Related papers  (2023-06-02T13:35:07Z)
• Behind Every Domain There is a Shift: Adapting Distortion-aware Vision Transformers for Panoramic Semantic Segmentation [73.48323921632506]
  We address panoramic semantic segmentation which is under-explored due to two critical challenges. First, we propose an upgraded Transformer for Panoramic Semantic, i.e., Trans4PASS+, equipped with Deformable Patch Embedding (DPE) and Deformable (DMLPv2) modules. Second, we enhance the Mutual Prototypical Adaptation (MPA) strategy via pseudo-label rectification for unsupervised domain adaptive panoramic segmentation. Third, aside from Pinhole-to-Panoramic (Pin2Pan) adaptation, we create a new dataset (SynPASS) with 9,080 panoramic images
  arXiv  Detail & Related papers  (2022-07-25T00:42:38Z)
• Bending Reality: Distortion-aware Transformers for Adapting to Panoramic Semantic Segmentation [26.09267582056609]
  Large quantities of expensive, pixel-wise annotations are crucial for success of robust panoramic segmentation models. Distortions and the distinct image-feature distribution in 360-degree panoramas impede the transfer from the annotation-rich pinhole domain. We learn object deformations and panoramic image distortions in Deformable Patch Embedding (DPE) and Deformable Deformable (DMLP) components. Finally, we tie together shared semantics in pinhole- and panoramic feature embeddings by generating multi-scale prototype features.
  arXiv  Detail & Related papers  (2022-03-02T23:00:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.