Contrastive Weighted Learning for Near-Infrared Gaze Estimation

• URL: http://arxiv.org/abs/2211.03073v1
• Date: Sun, 6 Nov 2022 10:03:23 GMT
• Title: Contrastive Weighted Learning for Near-Infrared Gaze Estimation
• Authors: Adam Lee
• Abstract summary: This paper proposes GazeCWL, a novel framework for gaze estimation with near-infrared images using contrastive learning. Our model outperforms previous domain generalization models in infrared image based gaze estimation.
• Score: 0.228438857884398
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Appearance-based gaze estimation has been very successful with the use of deep learning. Many following works improved domain generalization for gaze estimation. However, even though there has been much progress in domain generalization for gaze estimation, most of the recent work have been focused on cross-dataset performance -- accounting for different distributions in illuminations, head pose, and lighting. Although improving gaze estimation in different distributions of RGB images is important, near-infrared image based gaze estimation is also critical for gaze estimation in dark settings. Also there are inherent limitations relying solely on supervised learning for regression tasks. This paper contributes to solving these problems and proposes GazeCWL, a novel framework for gaze estimation with near-infrared images using contrastive learning. This leverages adversarial attack techniques for data augmentation and a novel contrastive loss function specifically for regression tasks that effectively clusters the features of different samples in the latent space. Our model outperforms previous domain generalization models in infrared image based gaze estimation and outperforms the baseline by 45.6\% while improving the state-of-the-art by 8.6\%, we demonstrate the efficacy of our method.

Related papers

• Data Augmentation via Latent Diffusion for Saliency Prediction [67.88936624546076]
  Saliency prediction models are constrained by the limited diversity and quantity of labeled data. We propose a novel data augmentation method for deep saliency prediction that edits natural images while preserving the complexity and variability of real-world scenes.
  arXiv  Detail & Related papers  (2024-09-11T14:36:24Z)
• Improving Neural Surface Reconstruction with Feature Priors from Multi-View Image [87.00660347447494]
  Recent advancements in Neural Surface Reconstruction (NSR) have significantly improved multi-view reconstruction when coupled with volume rendering. We propose an investigation into feature-level consistent loss, aiming to harness valuable feature priors from diverse pretext visual tasks. Our results, analyzed on DTU and EPFL, reveal that feature priors from image matching and multi-view stereo datasets outperform other pretext tasks.
  arXiv  Detail & Related papers  (2024-08-04T16:09:46Z)
• Semi-supervised Contrastive Regression for Estimation of Eye Gaze [0.609170287691728]
  This paper develops a semi-supervised contrastive learning framework for estimation of gaze direction. With a small labeled gaze dataset, the framework is able to find a generalized solution even for unseen face images. Our contrastive regression framework shows good performance in comparison to several state of the art contrastive learning techniques used for gaze estimation.
  arXiv  Detail & Related papers  (2023-08-05T04:11:38Z)
• 3DGazeNet: Generalizing Gaze Estimation with Weak-Supervision from Synthetic Views [67.00931529296788]
  We propose to train general gaze estimation models which can be directly employed in novel environments without adaptation. We create a large-scale dataset of diverse faces with gaze pseudo-annotations, which we extract based on the 3D geometry of the scene. We test our method in the task of gaze generalization, in which we demonstrate improvement of up to 30% compared to state-of-the-art when no ground truth data are available.
  arXiv  Detail & Related papers  (2022-12-06T14:15:17Z)
• Jitter Does Matter: Adapting Gaze Estimation to New Domains [12.482427155726413]
  We propose to utilize gaze jitter to analyze and optimize gaze domain adaptation task. We find that the high-frequency component (HFC) is an important factor that leads to jitter. We employ contrastive learning to encourage the model to obtain similar representations between original and perturbed data.
  arXiv  Detail & Related papers  (2022-10-05T08:20:41Z)
• Wild ToFu: Improving Range and Quality of Indirect Time-of-Flight Depth with RGB Fusion in Challenging Environments [56.306567220448684]
  We propose a new learning based end-to-end depth prediction network which takes noisy raw I-ToF signals as well as an RGB image. We show more than 40% RMSE improvement on the final depth map compared to the baseline approach.
  arXiv  Detail & Related papers  (2021-12-07T15:04:14Z)
• CAMERAS: Enhanced Resolution And Sanity preserving Class Activation Mapping for image saliency [61.40511574314069]
  Backpropagation image saliency aims at explaining model predictions by estimating model-centric importance of individual pixels in the input. We propose CAMERAS, a technique to compute high-fidelity backpropagation saliency maps without requiring any external priors.
  arXiv  Detail & Related papers  (2021-06-20T08:20:56Z)
• Non-Convex Tensor Low-Rank Approximation for Infrared Small Target Detection [32.67489082946838]
  Infrared small target detection plays an important role in many infrared systems. Most low-rank methods assign different singular values to inaccurate background estimation. We propose a non-native spatial approximation (NTLA) for this small infrared target detection algorithm.
  arXiv  Detail & Related papers  (2021-05-31T14:04:58Z)
• Perceptual Loss for Robust Unsupervised Homography Estimation [1.2891210250935146]
  BiHomE minimizes the distance in the feature space between the warped image from the source viewpoint and the corresponding image from the target viewpoint. We show that biHomE achieves state-of-the-art performance on synthetic COCO dataset, which is also comparable or better compared to supervised approaches.
  arXiv  Detail & Related papers  (2021-04-20T14:41:54Z)
• Vulnerability of Appearance-based Gaze Estimation [16.0559382707645]
  Appearance-based gaze estimation has achieved significant improvement by using deep learning. In this paper, we investigate the vulnerability of appearance-based gaze estimation. We show that the CA-Net shows superior performance against attack among the four popular appearance-based gaze estimation networks.
  arXiv  Detail & Related papers  (2021-03-24T12:19:59Z)
• Monte Carlo Dropout Ensembles for Robust Illumination Estimation [94.14796147340041]
  Computational color constancy is a preprocessing step used in many camera systems. We propose to aggregate different deep learning methods according to their output uncertainty. The proposed framework leads to state-of-the-art performance on INTEL-TAU dataset.
  arXiv  Detail & Related papers  (2020-07-20T13:56:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.