Unsupervised Enhancement of Real-World Depth Images Using Tri-Cycle GAN

• URL: http://arxiv.org/abs/2001.03779v1
• Date: Sat, 11 Jan 2020 18:19:09 GMT
• Title: Unsupervised Enhancement of Real-World Depth Images Using Tri-Cycle GAN
• Authors: Alona Baruhov and Guy Gilboa
• Abstract summary: We aim to enhance highly degraded, real-world depth images acquired by a low-cost sensor. In the absence of clean ground-truth, we approach the task as an unsupervised domain-translation between the low-quality sensor domain and a high-quality sensor domain. We employ the highly-successful Cycle-GAN to this task, but find it to perform poorly in this case.
• Score: 8.477619837043214
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Low quality depth poses a considerable challenge to computer vision algorithms. In this work we aim to enhance highly degraded, real-world depth images acquired by a low-cost sensor, for which an analytical noise model is unavailable. In the absence of clean ground-truth, we approach the task as an unsupervised domain-translation between the low-quality sensor domain and a high-quality sensor domain, represented using two unpaired training sets. We employ the highly-successful Cycle-GAN to this task, but find it to perform poorly in this case. Identifying the sources of the failure, we introduce several modifications to the framework, including a larger generator architecture, depth-specific losses that take into account missing pixels, and a novel Tri-Cycle loss which promotes information-preservation while addressing the asymmetry between the domains. We show that the resulting framework dramatically improves over the original Cycle-GAN both visually and quantitatively, extending its applicability to more challenging and asymmetric translation tasks.

Related papers

• AerialMegaDepth: Learning Aerial-Ground Reconstruction and View Synthesis [57.249817395828174]
  We propose a scalable framework combining pseudo-synthetic renderings from 3D city-wide meshes with real, ground-level crowd-sourced images. The pseudo-synthetic data simulates a wide range of aerial viewpoints, while the real, crowd-sourced images help improve visual fidelity for ground-level images. Using this hybrid dataset, we fine-tune several state-of-the-art algorithms and achieve significant improvements on real-world, zero-shot aerial-ground tasks.
  arXiv  Detail & Related papers  (2025-04-17T17:57:05Z)
• Overcoming the Identity Mapping Problem in Self-Supervised Hyperspectral Anomaly Detection [6.547234350675682]
  Self-supervised Hyperspectral Anomaly Detection (HAD) models often suffer from the Identity Mapping Problem (IMP) IMP manifests as a tendency for the model to overfit to the entire image, particularly with increasing network complexity or prolonged training iterations. We introduce three solutions: superpixel pooling and uppooling for perturbation, error-adaptive convolution for reconstruction, and online background pixel mining for regularization.
  arXiv  Detail & Related papers  (2025-04-05T09:12:25Z)
• HazyDet: Open-Source Benchmark for Drone-View Object Detection with Depth-Cues in Hazy Scenes [54.24350833692194]
  HazyDet is the first, large-scale benchmark specifically designed for drone-view object detection in hazy conditions.<n>We propose the Depth-Conditioned Detector (DeCoDet) to address the severe visual degradation induced by haze.<n>HazyDet provides a challenging and realistic testbed for advancing detection algorithms.
  arXiv  Detail & Related papers  (2024-09-30T00:11:40Z)
• Robust Disaster Assessment from Aerial Imagery Using Text-to-Image Synthetic Data [66.49494950674402]
  We leverage emerging text-to-image generative models in creating large-scale synthetic supervision for the task of damage assessment from aerial images. We build an efficient and easily scalable pipeline to generate thousands of post-disaster images from low-resource domains. We validate the strength of our proposed framework under cross-geography domain transfer setting from xBD and SKAI images in both single-source and multi-source settings.
  arXiv  Detail & Related papers  (2024-05-22T16:07:05Z)
• GEOcc: Geometrically Enhanced 3D Occupancy Network with Implicit-Explicit Depth Fusion and Contextual Self-Supervision [49.839374549646884]
  This paper presents GEOcc, a Geometric-Enhanced Occupancy network tailored for vision-only surround-view perception. Our approach achieves State-Of-The-Art performance on the Occ3D-nuScenes dataset with the least image resolution needed and the most weightless image backbone.
  arXiv  Detail & Related papers  (2024-05-17T07:31:20Z)
• Multi-Sensor Diffusion-Driven Optical Image Translation for Large-Scale Applications [3.4085512042262374]
  We present a novel method that super-resolves large-scale low spatial resolution images into high-resolution equivalents from disparate optical sensors. Our contributions lie in new forward and reverse diffusion processes, which are crucial for addressing the challenges of large-scale image translation. The outcome is a high-resolution large-scale image with consistent patches, vital for applications such as heterogeneous change detection.
  arXiv  Detail & Related papers  (2024-04-17T10:49:00Z)
• ReContrast: Domain-Specific Anomaly Detection via Contrastive Reconstruction [29.370142078092375]
  Most advanced unsupervised anomaly detection (UAD) methods rely on modeling feature representations of frozen encoder networks pre-trained on large-scale datasets. We propose a novel epistemic UAD method, namely ReContrast, which optimize the entire network to reduce biases towards the pre-trained image domain. We conduct experiments across two popular industrial defect detection benchmarks and three medical image UAD tasks, which shows our superiority over current state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-06-05T05:21:15Z)
• Manifold Constraint Regularization for Remote Sensing Image Generation [34.68714863219855]
  Generative Adversarial Networks (GANs) have shown notable accomplishments in remote sensing domain. This paper analyzes the characteristics of remote sensing images and proposes manifold constraint regularization.
  arXiv  Detail & Related papers  (2023-05-31T02:35:41Z)
• Towards Domain Generalization for Multi-view 3D Object Detection in Bird-Eye-View [11.958753088613637]
  We first analyze the causes of the domain gap for the MV3D-Det task. To acquire a robust depth prediction, we propose to decouple the depth estimation from intrinsic parameters of the camera. We modify the focal length values to create multiple pseudo-domains and construct an adversarial training loss to encourage the feature representation to be more domain-agnostic.
  arXiv  Detail & Related papers  (2023-03-03T02:59:13Z)
• Adversarial Domain Feature Adaptation for Bronchoscopic Depth Estimation [111.89519571205778]
  In this work, we propose an alternative domain-adaptive approach to depth estimation. Our novel two-step structure first trains a depth estimation network with labeled synthetic images in a supervised manner. The results of our experiments show that the proposed method improves the network's performance on real images by a considerable margin.
  arXiv  Detail & Related papers  (2021-09-24T08:11:34Z)
• Spatially-Adaptive Image Restoration using Distortion-Guided Networks [51.89245800461537]
  We present a learning-based solution for restoring images suffering from spatially-varying degradations. We propose SPAIR, a network design that harnesses distortion-localization information and dynamically adjusts to difficult regions in the image.
  arXiv  Detail & Related papers  (2021-08-19T11:02:25Z)
• Unsupervised Deep Image Stitching: Reconstructing Stitched Features to Images [38.95610086309832]
  We propose an unsupervised deep image stitching framework consisting of two stages: unsupervised coarse image alignment and unsupervised image reconstruction. In the first stage, we design an ablation-based loss to constrain an unsupervised homography network, which is more suitable for large-baseline scenes. In the second stage, motivated by the insight that the misalignments in pixel-level can be eliminated to a certain extent in feature-level, we design an unsupervised image reconstruction network to eliminate the artifacts from features to pixels.
  arXiv  Detail & Related papers  (2021-06-24T09:45:36Z)
• Gated Fusion Network for Degraded Image Super Resolution [78.67168802945069]
  We propose a dual-branch convolutional neural network to extract base features and recovered features separately. By decomposing the feature extraction step into two task-independent streams, the dual-branch model can facilitate the training process.
  arXiv  Detail & Related papers  (2020-03-02T13:28:32Z)
• Image Fine-grained Inpainting [89.17316318927621]
  We present a one-stage model that utilizes dense combinations of dilated convolutions to obtain larger and more effective receptive fields. To better train this efficient generator, except for frequently-used VGG feature matching loss, we design a novel self-guided regression loss. We also employ a discriminator with local and global branches to ensure local-global contents consistency.
  arXiv  Detail & Related papers  (2020-02-07T03:45:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.