Jitter Does Matter: Adapting Gaze Estimation to New Domains

• URL: http://arxiv.org/abs/2210.02082v1
• Date: Wed, 5 Oct 2022 08:20:41 GMT
• Title: Jitter Does Matter: Adapting Gaze Estimation to New Domains
• Authors: Ruicong Liu, Yiwei Bao, Mingjie Xu, Haofei Wang, Yunfei Liu, Feng Lu
• Abstract summary: We propose to utilize gaze jitter to analyze and optimize gaze domain adaptation task. We find that the high-frequency component (HFC) is an important factor that leads to jitter. We employ contrastive learning to encourage the model to obtain similar representations between original and perturbed data.
• Score: 12.482427155726413
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep neural networks have demonstrated superior performance on appearance-based gaze estimation tasks. However, due to variations in person, illuminations, and background, performance degrades dramatically when applying the model to a new domain. In this paper, we discover an interesting gaze jitter phenomenon in cross-domain gaze estimation, i.e., the gaze predictions of two similar images can be severely deviated in target domain. This is closely related to cross-domain gaze estimation tasks, but surprisingly, it has not been noticed yet previously. Therefore, we innovatively propose to utilize the gaze jitter to analyze and optimize the gaze domain adaptation task. We find that the high-frequency component (HFC) is an important factor that leads to jitter. Based on this discovery, we add high-frequency components to input images using the adversarial attack and employ contrastive learning to encourage the model to obtain similar representations between original and perturbed data, which reduces the impacts of HFC. We evaluate the proposed method on four cross-domain gaze estimation tasks, and experimental results demonstrate that it significantly reduces the gaze jitter and improves the gaze estimation performance in target domains.

Related papers

• Understanding and Improving Training-Free AI-Generated Image Detections with Vision Foundation Models [68.90917438865078]
  Deepfake techniques for facial synthesis and editing pose serious risks for generative models. In this paper, we investigate how detection performance varies across model backbones, types, and datasets. We introduce Contrastive Blur, which enhances performance on facial images, and MINDER, which addresses noise type bias, balancing performance across domains.
  arXiv  Detail & Related papers  (2024-11-28T13:04:45Z)
• Diffusion-Based Particle-DETR for BEV Perception [94.88305708174796]
  Bird-Eye-View (BEV) is one of the most widely-used scene representations for visual perception in Autonomous Vehicles (AVs) Recent diffusion-based methods offer a promising approach to uncertainty modeling for visual perception but fail to effectively detect small objects in the large coverage of the BEV. Here, we address this problem by combining the diffusion paradigm with current state-of-the-art 3D object detectors in BEV.
  arXiv  Detail & Related papers  (2023-12-18T09:52:14Z)
• Adaptive Face Recognition Using Adversarial Information Network [57.29464116557734]
  Face recognition models often degenerate when training data are different from testing data. We propose a novel adversarial information network (AIN) to address it.
  arXiv  Detail & Related papers  (2023-05-23T02:14:11Z)
• Contrastive Weighted Learning for Near-Infrared Gaze Estimation [0.228438857884398]
  This paper proposes GazeCWL, a novel framework for gaze estimation with near-infrared images using contrastive learning. Our model outperforms previous domain generalization models in infrared image based gaze estimation.
  arXiv  Detail & Related papers  (2022-11-06T10:03:23Z)
• LatentGaze: Cross-Domain Gaze Estimation through Gaze-Aware Analytic Latent Code Manipulation [0.0]
  We propose a gaze-aware analytic manipulation method, based on a data-driven approach with generative adversarial network inversion's disentanglement characteristics. By utilizing GAN-based encoder-generator process, we shift the input image from the target domain to the source domain image, which a gaze estimator is sufficiently aware.
  arXiv  Detail & Related papers  (2022-09-21T08:05:53Z)
• Active Gaze Control for Foveal Scene Exploration [124.11737060344052]
  We propose a methodology to emulate how humans and robots with foveal cameras would explore a scene. The proposed method achieves an increase in detection F1-score of 2-3 percentage points for the same number of gaze shifts.
  arXiv  Detail & Related papers  (2022-08-24T14:59:28Z)
• Unsupervised Compound Domain Adaptation for Face Anti-Spoofing [74.6122128643823]
  We address the problem of face anti-spoofing which aims to make the face verification systems robust in the real world settings. We propose a memory augmentation method for adapting the source model to the target domain in a domain aware manner. The proposed method successfully adapts to the compound target domain consisting multiple new spoof types.
  arXiv  Detail & Related papers  (2021-05-18T12:08:07Z)
• An Adversarial Human Pose Estimation Network Injected with Graph Structure [75.08618278188209]
  In this paper, we design a novel generative adversarial network (GAN) to improve the localization accuracy of visible joints when some joints are invisible. The network consists of two simple but efficient modules, Cascade Feature Network (CFN) and Graph Structure Network (GSN)
  arXiv  Detail & Related papers  (2021-03-29T12:07:08Z)
• PureGaze: Purifying Gaze Feature for Generalizable Gaze Estimation [12.076469954457007]
  We tackle the domain generalization problem in cross-domain gaze estimation for unknown target domains. To be specific, we realize the domain generalization by gaze feature purification. We design a plug-and-play self-adversarial framework for the gaze feature purification.
  arXiv  Detail & Related papers  (2021-03-24T13:22:00Z)
• Vulnerability of Appearance-based Gaze Estimation [16.0559382707645]
  Appearance-based gaze estimation has achieved significant improvement by using deep learning. In this paper, we investigate the vulnerability of appearance-based gaze estimation. We show that the CA-Net shows superior performance against attack among the four popular appearance-based gaze estimation networks.
  arXiv  Detail & Related papers  (2021-03-24T12:19:59Z)
• Domain Adaptation Gaze Estimation by Embedding with Prediction Consistency [10.246471430786244]
  This paper proposes an unsupervised method for domain adaptation gaze estimation. We employ source gaze to form a locally linear representation in the gaze space for each target domain prediction. The same linear combinations are applied in the embedding space to generate hypothesis embedding for the target domain sample.
  arXiv  Detail & Related papers  (2020-11-15T13:33:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.