Gaze Preserving CycleGANs for Eyeglass Removal & Persistent Gaze Estimation

• URL: http://arxiv.org/abs/2002.02077v6
• Date: Tue, 15 Jun 2021 21:41:54 GMT
• Title: Gaze Preserving CycleGANs for Eyeglass Removal & Persistent Gaze Estimation
• Authors: Akshay Rangesh, Bowen Zhang and Mohan M. Trivedi
• Abstract summary: Estimating the gaze direction is the most obvious way to gauge a driver's state under ideal conditions. Relying on head pose alone under harsh illumination, nighttime conditions, and reflective eyeglasses can prove to be unreliable and erroneous. Our proposed Gaze Preserving CycleGAN (GPCycleGAN) is trained to preserve the driver's gaze while removing potential eyeglasses from face images.
• Score: 8.47514372451741
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A driver's gaze is critical for determining their attention, state, situational awareness, and readiness to take over control from partially automated vehicles. Estimating the gaze direction is the most obvious way to gauge a driver's state under ideal conditions when limited to using non-intrusive imaging sensors. Unfortunately, the vehicular environment introduces a variety of challenges that are usually unaccounted for - harsh illumination, nighttime conditions, and reflective eyeglasses. Relying on head pose alone under such conditions can prove to be unreliable and erroneous. In this study, we offer solutions to address these problems encountered in the real world. To solve issues with lighting, we demonstrate that using an infrared camera with suitable equalization and normalization suffices. To handle eyeglasses and their corresponding artifacts, we adopt image-to-image translation using generative adversarial networks to pre-process images prior to gaze estimation. Our proposed Gaze Preserving CycleGAN (GPCycleGAN) is trained to preserve the driver's gaze while removing potential eyeglasses from face images. GPCycleGAN is based on the well-known CycleGAN approach - with the addition of a gaze classifier and a gaze consistency loss for additional supervision. Our approach exhibits improved performance, interpretability, robustness and superior qualitative results on challenging real-world datasets.

Related papers

• OOSTraj: Out-of-Sight Trajectory Prediction With Vision-Positioning Denoising [49.86409475232849]
  Trajectory prediction is fundamental in computer vision and autonomous driving. Existing approaches in this field often assume precise and complete observational data. We present a novel method for out-of-sight trajectory prediction that leverages a vision-positioning technique.
  arXiv  Detail & Related papers  (2024-04-02T18:30:29Z)
• What Do You See in Vehicle? Comprehensive Vision Solution for In-Vehicle Gaze Estimation [18.155092199205907]
  In this paper, we present three novel elements to advance in-vehicle gaze research. First, we introduce IVGaze, a pioneering dataset capturing in-vehicle gaze. Second, our research focuses on in-vehicle gaze estimation leveraging the IVGaze. Third, we explore a novel strategy for gaze zone classification by extending the GazeDPTR.
  arXiv  Detail & Related papers  (2024-03-23T01:22:15Z)
• Learning to Find Missing Video Frames with Synthetic Data Augmentation: A General Framework and Application in Generating Thermal Images Using RGB Cameras [0.0]
  This paper addresses the issue of missing data due to sensor frame rate mismatches. We propose using conditional generative adversarial networks (cGANs) to create synthetic yet realistic thermal imagery.
  arXiv  Detail & Related papers  (2024-02-29T23:52:15Z)
• NiteDR: Nighttime Image De-Raining with Cross-View Sensor Cooperative Learning for Dynamic Driving Scenes [49.92839157944134]
  In nighttime driving scenes, insufficient and uneven lighting shrouds the scenes in darkness, resulting degradation of image quality and visibility. We develop an image de-raining framework tailored for rainy nighttime driving scenes. It aims to remove rain artifacts, enrich scene representation, and restore useful information.
  arXiv  Detail & Related papers  (2024-02-28T09:02:33Z)
• Generating Clear Images From Images With Distortions Caused by Adverse Weather Using Generative Adversarial Networks [0.0]
  We presented a method for improving computer vision tasks on images affected by adverse weather conditions, including distortions caused by adherent raindrops. We trained an appropriate generative adversarial network and showed that it was effective at removing the effect of the distortions.
  arXiv  Detail & Related papers  (2022-11-01T05:02:44Z)
• Thermal to Visible Image Synthesis under Atmospheric Turbulence [67.99407460140263]
  In biometrics and surveillance, thermal imagining modalities are often used to capture images in low-light and nighttime conditions. Such imaging systems often suffer from atmospheric turbulence, which introduces severe blur and deformation artifacts to the captured images. An end-to-end reconstruction method is proposed which can directly transform thermal images into visible-spectrum images.
  arXiv  Detail & Related papers  (2022-04-06T19:47:41Z)
• DMRVisNet: Deep Multi-head Regression Network for Pixel-wise Visibility Estimation Under Foggy Weather [0.0]
  Fog, as a kind of common weather, frequently appears in the real world, especially in the mountain areas. Current methods use professional instruments outfitted at fixed locations on the roads to perform the visibility measurement. We propose an innovative end-to-end convolutional neural network framework to estimate the visibility.
  arXiv  Detail & Related papers  (2021-12-08T13:31:07Z)
• Simultaneous Face Hallucination and Translation for Thermal to Visible Face Verification using Axial-GAN [74.22129648654783]
  We introduce the task of thermal-to-visible face verification from low-resolution thermal images. We propose Axial-Generative Adversarial Network (Axial-GAN) to synthesize high-resolution visible images for matching.
  arXiv  Detail & Related papers  (2021-04-13T22:34:28Z)
• Calibrating Self-supervised Monocular Depth Estimation [77.77696851397539]
  In the recent years, many methods demonstrated the ability of neural networks to learn depth and pose changes in a sequence of images, using only self-supervision as the training signal. We show that incorporating prior information about the camera configuration and the environment, we can remove the scale ambiguity and predict depth directly, still using the self-supervised formulation and not relying on any additional sensors.
  arXiv  Detail & Related papers  (2020-09-16T14:35:45Z)
• A feature-supervised generative adversarial network for environmental monitoring during hazy days [6.276954295407201]
  This paper proposes a feature-supervised learning network based on generative adversarial networks (GAN) for environmental monitoring during days. The proposed method has achieved better performance than current state-of-the-art methods on both synthetic datasets and real-world remote sensing images.
  arXiv  Detail & Related papers  (2020-08-05T05:27:15Z)
• Exploring Thermal Images for Object Detection in Underexposure Regions for Autonomous Driving [67.69430435482127]
  Underexposure regions are vital to construct a complete perception of the surroundings for safe autonomous driving. The availability of thermal cameras has provided an essential alternate to explore regions where other optical sensors lack in capturing interpretable signals. This work proposes a domain adaptation framework which employs a style transfer technique for transfer learning from visible spectrum images to thermal images.
  arXiv  Detail & Related papers  (2020-06-01T09:59:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.