A Fixation-based 360{\deg} Benchmark Dataset for Salient Object Detection

• URL: http://arxiv.org/abs/2001.07960v2
• Date: Tue, 19 May 2020 15:26:21 GMT
• Title: A Fixation-based 360{\deg} Benchmark Dataset for Salient Object Detection
• Authors: Yi Zhang, Lu Zhang, Wassim Hamidouche, Olivier Deforges
• Abstract summary: Fixation prediction (FP) in panoramic contents has been widely investigated along with the booming trend of virtual reality (VR) applications. salient object detection (SOD) has been seldom explored in 360deg images due to the lack of datasets representative of real scenes.
• Score: 21.314578493964333
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Fixation prediction (FP) in panoramic contents has been widely investigated along with the booming trend of virtual reality (VR) applications. However, another issue within the field of visual saliency, salient object detection (SOD), has been seldom explored in 360{\deg} (or omnidirectional) images due to the lack of datasets representative of real scenes with pixel-level annotations. Toward this end, we collect 107 equirectangular panoramas with challenging scenes and multiple object classes. Based on the consistency between FP and explicit saliency judgements, we further manually annotate 1,165 salient objects over the collected images with precise masks under the guidance of real human eye fixation maps. Six state-of-the-art SOD models are then benchmarked on the proposed fixation-based 360{\deg} image dataset (F-360iSOD), by applying a multiple cubic projection-based fine-tuning method. Experimental results show a limitation of the current methods when used for SOD in panoramic images, which indicates the proposed dataset is challenging. Key issues for 360{\deg} SOD is also discussed. The proposed dataset is available at https://github.com/PanoAsh/F-360iSOD.

Related papers

• Sp2360: Sparse-view 360 Scene Reconstruction using Cascaded 2D Diffusion Priors [51.36238367193988]
  We tackle sparse-view reconstruction of a 360 3D scene using priors from latent diffusion models (LDM) We present SparseSplat360, a method that employs a cascade of in-painting and artifact removal models to fill in missing details and clean novel views. Our method generates entire 360 scenes from as few as 9 input views, with a high degree of foreground and background detail.
  arXiv  Detail & Related papers  (2024-05-26T11:01:39Z)
• 360VOTS: Visual Object Tracking and Segmentation in Omnidirectional Videos [16.372814014632944]
  We propose a comprehensive dataset and benchmark that incorporates a new component called omnidirectional video object segmentation (360VOS) 360VOS dataset includes 290 sequences accompanied by dense pixel-wise masks and covers a broader range of target categories. We benchmark state-of-the-art approaches and demonstrate the effectiveness of our proposed 360 tracking framework and training dataset.
  arXiv  Detail & Related papers  (2024-04-22T07:54:53Z)
• 360VOT: A New Benchmark Dataset for Omnidirectional Visual Object Tracking [10.87309734945868]
  360deg images can provide an omnidirectional field of view which is important for stable and long-term scene perception. In this paper, we explore 360deg images for visual object tracking and perceive new challenges caused by large distortion. We propose a new large-scale omnidirectional tracking benchmark dataset, 360VOT, in order to facilitate future research.
  arXiv  Detail & Related papers  (2023-07-27T05:32:01Z)
• Multi-Projection Fusion and Refinement Network for Salient Object Detection in 360{\deg} Omnidirectional Image [141.10227079090419]
  We propose a Multi-Projection Fusion and Refinement Network (MPFR-Net) to detect the salient objects in 360deg omnidirectional image. MPFR-Net uses the equirectangular projection image and four corresponding cube-unfolding images as inputs. Experimental results on two omnidirectional datasets demonstrate that the proposed approach outperforms the state-of-the-art methods both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2022-12-23T14:50:40Z)
• View-aware Salient Object Detection for 360{\deg} Omnidirectional Image [33.43250302656753]
  We construct a large scale 360deg ISOD dataset with object-level pixel-wise annotation on equirectangular projection (ERP) Inspired by humans' observing process, we propose a view-aware salient object detection method based on a Sample Adaptive View Transformer (SAVT) module.
  arXiv  Detail & Related papers  (2022-09-27T07:44:08Z)
• Scatter Points in Space: 3D Detection from Multi-view Monocular Images [8.71944437852952]
  3D object detection from monocular image(s) is a challenging and long-standing problem of computer vision. Recent methods tend to aggregate multiview feature by sampling regular 3D grid densely in space. We propose a learnable keypoints sampling method, which scatters pseudo surface points in 3D space, in order to keep data sparsity.
  arXiv  Detail & Related papers  (2022-08-31T09:38:05Z)
• Field-of-View IoU for Object Detection in 360{\deg} Images [36.72543749626039]
  We propose two fundamental techniques -- Field-of-View IoU (FoV-IoU) and 360Augmentation for object detection in 360deg images. FoV-IoU computes the intersection-over-union of two Field-of-View bounding boxes in a spherical image which could be used for training, inference, and evaluation. 360Augmentation is a data augmentation technique specific to 360deg object detection task which randomly rotates a spherical image and solves the bias due to the sphere-to-plane projection.
  arXiv  Detail & Related papers  (2022-02-07T14:01:59Z)
• VPFNet: Improving 3D Object Detection with Virtual Point based LiDAR and Stereo Data Fusion [62.24001258298076]
  VPFNet is a new architecture that cleverly aligns and aggregates the point cloud and image data at the virtual' points. Our VPFNet achieves 83.21% moderate 3D AP and 91.86% moderate BEV AP on the KITTI test set, ranking the 1st since May 21th, 2021.
  arXiv  Detail & Related papers  (2021-11-29T08:51:20Z)
• Salient Objects in Clutter [130.63976772770368]
  This paper identifies and addresses a serious design bias of existing salient object detection (SOD) datasets. This design bias has led to a saturation in performance for state-of-the-art SOD models when evaluated on existing datasets. We propose a new high-quality dataset and update the previous saliency benchmark.
  arXiv  Detail & Related papers  (2021-05-07T03:49:26Z)
• Single View Metrology in the Wild [94.7005246862618]
  We present a novel approach to single view metrology that can recover the absolute scale of a scene represented by 3D heights of objects or camera height above the ground. Our method relies on data-driven priors learned by a deep network specifically designed to imbibe weakly supervised constraints from the interplay of the unknown camera with 3D entities such as object heights. We demonstrate state-of-the-art qualitative and quantitative results on several datasets as well as applications including virtual object insertion.
  arXiv  Detail & Related papers  (2020-07-18T22:31:33Z)
• Visual Question Answering on 360{\deg} Images [96.00046925811515]
  VQA 360 is a novel task of visual question answering on 360 images. We collect the first VQA 360 dataset, containing around 17,000 real-world image-question-answer triplets for a variety of question types.
  arXiv  Detail & Related papers  (2020-01-10T08:18:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.