3D Visual Illusion Depth Estimation

• URL: http://arxiv.org/abs/2505.13061v3
• Date: Sat, 24 May 2025 07:37:09 GMT
• Title: 3D Visual Illusion Depth Estimation
• Authors: Chengtang Yao, Zhidan Liu, Jiaxi Zeng, Lidong Yu, Yuwei Wu, Yunde Jia,
• Abstract summary: 3D visual illusion is a perceptual phenomenon where a two-dimensional plane is manipulated to simulate three-dimensional spatial relationships.<n>In this paper, we reveal that the machine visual system is also seriously fooled by 3D visual illusions, including monocular and binocular depth estimation.
• Score: 27.15757281613792
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: 3D visual illusion is a perceptual phenomenon where a two-dimensional plane is manipulated to simulate three-dimensional spatial relationships, making a flat artwork or object look three-dimensional in the human visual system. In this paper, we reveal that the machine visual system is also seriously fooled by 3D visual illusions, including monocular and binocular depth estimation. In order to explore and analyze the impact of 3D visual illusion on depth estimation, we collect a large dataset containing almost 3k scenes and 200k images to train and evaluate SOTA monocular and binocular depth estimation methods. We also propose a robust depth estimation framework that uses common sense from a vision-language model to adaptively select reliable depth from binocular disparity and monocular depth. Experiments show that SOTA monocular, binocular, and multi-view depth estimation approaches are all fooled by various 3D visual illusions, while our method achieves SOTA performance.

Related papers

• Ross3D: Reconstructive Visual Instruction Tuning with 3D-Awareness [73.72335146374543]
  We introduce reconstructive visual instruction tuning with 3D-awareness (Ross3D), which integrates 3D-aware visual supervision into the training procedure.<n>Ross3D achieves state-of-the-art performance across various 3D scene understanding benchmarks.
  arXiv  Detail & Related papers  (2025-04-02T16:59:55Z)
• Weakly Supervised Monocular 3D Detection with a Single-View Image [58.57978772009438]
  Monocular 3D detection aims for precise 3D object localization from a single-view image. We propose SKD-WM3D, a weakly supervised monocular 3D detection framework. We show that SKD-WM3D surpasses the state-of-the-art clearly and is even on par with many fully supervised methods.
  arXiv  Detail & Related papers  (2024-02-29T13:26:47Z)
• 3D Concept Learning and Reasoning from Multi-View Images [96.3088005719963]
  We introduce a new large-scale benchmark for 3D multi-view visual question answering (3DMV-VQA) This dataset consists of approximately 5k scenes, 600k images, paired with 50k questions. We propose a novel 3D concept learning and reasoning framework that seamlessly combines neural fields, 2D pre-trained vision-language models, and neural reasoning operators.
  arXiv  Detail & Related papers  (2023-03-20T17:59:49Z)
• Monocular 3D Object Detection with Depth from Motion [74.29588921594853]
  We take advantage of camera ego-motion for accurate object depth estimation and detection. Our framework, named Depth from Motion (DfM), then uses the established geometry to lift 2D image features to the 3D space and detects 3D objects thereon. Our framework outperforms state-of-the-art methods by a large margin on the KITTI benchmark.
  arXiv  Detail & Related papers  (2022-07-26T15:48:46Z)
• Learning Ego 3D Representation as Ray Tracing [42.400505280851114]
  We present a novel end-to-end architecture for ego 3D representation learning from unconstrained camera views. Inspired by the ray tracing principle, we design a polarized grid of "imaginary eyes" as the learnable ego 3D representation. We show that our model outperforms all state-of-the-art alternatives significantly.
  arXiv  Detail & Related papers  (2022-06-08T17:55:50Z)
• Learning Geometry-Guided Depth via Projective Modeling for Monocular 3D Object Detection [70.71934539556916]
  We learn geometry-guided depth estimation with projective modeling to advance monocular 3D object detection. Specifically, a principled geometry formula with projective modeling of 2D and 3D depth predictions in the monocular 3D object detection network is devised. Our method remarkably improves the detection performance of the state-of-the-art monocular-based method without extra data by 2.80% on the moderate test setting.
  arXiv  Detail & Related papers  (2021-07-29T12:30:39Z)
• Disentangling and Vectorization: A 3D Visual Perception Approach for Autonomous Driving Based on Surround-View Fisheye Cameras [3.485767750936058]
  Multidimensional Vector is proposed to include the utilizable information generated in different dimensions and stages. The experiments of real fisheye images demonstrate that our solution achieves state-of-the-art accuracy while being real-time in practice.
  arXiv  Detail & Related papers  (2021-07-19T13:24:21Z)
• MonoGRNet: A General Framework for Monocular 3D Object Detection [23.59839921644492]
  We propose MonoGRNet for the amodal 3D object detection from a monocular image via geometric reasoning. MonoGRNet decomposes the monocular 3D object detection task into four sub-tasks including 2D object detection, instance-level depth estimation, projected 3D center estimation and local corner regression. Experiments are conducted on KITTI, Cityscapes and MS COCO datasets.
  arXiv  Detail & Related papers  (2021-04-18T10:07:52Z)
• Monocular Differentiable Rendering for Self-Supervised 3D Object Detection [21.825158925459732]
  3D object detection from monocular images is an ill-posed problem due to the projective entanglement of depth and scale. We present a novel self-supervised method for textured 3D shape reconstruction and pose estimation of rigid objects. Our method predicts the 3D location and meshes of each object in an image using differentiable rendering and a self-supervised objective.
  arXiv  Detail & Related papers  (2020-09-30T09:21:43Z)
• 3D Crowd Counting via Geometric Attention-guided Multi-View Fusion [50.520192402702015]
  We propose to solve the multi-view crowd counting task through 3D feature fusion with 3D scene-level density maps. Compared to 2D fusion, the 3D fusion extracts more information of the people along the z-dimension (height), which helps to address the scale variations across multiple views. The 3D density maps still preserve the 2D density maps property that the sum is the count, while also providing 3D information about the crowd density.
  arXiv  Detail & Related papers  (2020-03-18T11:35:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.