STEREOFOG -- Computational DeFogging via Image-to-Image Translation on a real-world Dataset

• URL: http://arxiv.org/abs/2312.02344v1
• Date: Mon, 4 Dec 2023 21:07:13 GMT
• Title: STEREOFOG -- Computational DeFogging via Image-to-Image Translation on a real-world Dataset
• Authors: Anton Pollak, Rajesh Menon
• Abstract summary: Image-to-Image translation (I2I) is a subtype of Machine Learning (ML) that has tremendous potential in applications. We introduce STEREOFOG, a dataset comprised of $10,067$ paired fogged and clear images. We apply and optimize the pix2pix I2I ML framework to this dataset.
• Score: 0.8702432681310401
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Image-to-Image translation (I2I) is a subtype of Machine Learning (ML) that has tremendous potential in applications where two domains of images and the need for translation between the two exist, such as the removal of fog. For example, this could be useful for autonomous vehicles, which currently struggle with adverse weather conditions like fog. However, datasets for I2I tasks are not abundant and typically hard to acquire. Here, we introduce STEREOFOG, a dataset comprised of $10,067$ paired fogged and clear images, captured using a custom-built device, with the purpose of exploring I2I's potential in this domain. It is the only real-world dataset of this kind to the best of our knowledge. Furthermore, we apply and optimize the pix2pix I2I ML framework to this dataset. With the final model achieving an average Complex Wavelet-Structural Similarity (CW-SSIM) score of $0.76$, we prove the technique's suitability for the problem.

Related papers

• Implicit-Zoo: A Large-Scale Dataset of Neural Implicit Functions for 2D Images and 3D Scenes [65.22070581594426]
  "Implicit-Zoo" is a large-scale dataset requiring thousands of GPU training days to facilitate research and development in this field. We showcase two immediate benefits as it enables to: (1) learn token locations for transformer models; (2) directly regress 3D cameras poses of 2D images with respect to NeRF models. This in turn leads to an improved performance in all three task of image classification, semantic segmentation, and 3D pose regression, thereby unlocking new avenues for research.
  arXiv  Detail & Related papers  (2024-06-25T10:20:44Z)
• CoFiI2P: Coarse-to-Fine Correspondences for Image-to-Point Cloud Registration [9.57539651520755]
  CoFiI2P is a novel I2P registration network that extracts correspondences in a coarse-to-fine manner. In the coarse matching phase, a novel I2P transformer module is employed to capture both homogeneous and heterogeneous global information. In the fine matching module, point/pixel pairs are established with the guidance of super-point/super-pixel correspondences.
  arXiv  Detail & Related papers  (2023-09-26T04:32:38Z)
• DiffI2I: Efficient Diffusion Model for Image-to-Image Translation [108.82579440308267]
  Diffusion Model (DM) has emerged as the SOTA approach for image synthesis. DM can't perform well on some image-to-image translation (I2I) tasks. DiffI2I comprises three key components: a compact I2I prior extraction network (CPEN), a dynamic I2I transformer (DI2Iformer) and a denoising network.
  arXiv  Detail & Related papers  (2023-08-26T05:18:23Z)
• I2P-Rec: Recognizing Images on Large-scale Point Cloud Maps through Bird's Eye View Projections [18.7557037030769]
  Place recognition is an important technique for autonomous cars to achieve full autonomy. We propose the I2P-Rec method to solve the problem by transforming the cross-modal data into the same modality. With only a small set of training data, I2P-Rec achieves recall rates at Top-1% over 80% and 90%, when localizing monocular and stereo images on point cloud maps.
  arXiv  Detail & Related papers  (2023-03-02T07:56:04Z)
• {\mu}Split: efficient image decomposition for microscopy data [50.794670705085835]
  muSplit is a dedicated approach for trained image decomposition in the context of fluorescence microscopy images. We introduce lateral contextualization (LC), a novel meta-architecture that enables the memory efficient incorporation of large image-context. We apply muSplit to five decomposition tasks, one on a synthetic dataset, four others derived from real microscopy data.
  arXiv  Detail & Related papers  (2022-11-23T11:26:24Z)
• TransferI2I: Transfer Learning for Image-to-Image Translation from Small Datasets [35.84311497205075]
  Image-to-image (I2I) translation has matured in recent years and is able to generate high-quality realistic images. Existing methods use transfer learning for I2I translation, but they still require the learning of millions of parameters from scratch. We propose a new transfer learning for I2I translation (TransferI2I)
  arXiv  Detail & Related papers  (2021-05-13T12:19:36Z)
• Unsupervised Image-to-Image Translation via Pre-trained StyleGAN2 Network [73.5062435623908]
  We propose a new I2I translation method that generates a new model in the target domain via a series of model transformations. By feeding the latent vector into the generated model, we can perform I2I translation between the source domain and target domain.
  arXiv  Detail & Related papers  (2020-10-12T13:51:40Z)
• Analogical Image Translation for Fog Generation [114.39308837759329]
  AIT learns with synthetic clear-weather images, synthetic foggy images and real clear-weather images to add fog effects onto real clear-weather images without seeing any real foggy images during training. AIT achieves this zero-shot image translation capability by coupling a supervised training scheme in the synthetic domain, a cycle consistency strategy in the real domain, an adversarial training scheme between the two domains, and a novel network design.
  arXiv  Detail & Related papers  (2020-06-28T14:33:31Z)
• TuiGAN: Learning Versatile Image-to-Image Translation with Two Unpaired Images [102.4003329297039]
  An unsupervised image-to-image translation (UI2I) task deals with learning a mapping between two domains without paired images. We propose TuiGAN, a generative model that is trained on only two unpaired images and amounts to one-shot unsupervised learning.
  arXiv  Detail & Related papers  (2020-04-09T16:23:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.