PureGaze: Purifying Gaze Feature for Generalizable Gaze Estimation
- URL: http://arxiv.org/abs/2103.13173v1
- Date: Wed, 24 Mar 2021 13:22:00 GMT
- Title: PureGaze: Purifying Gaze Feature for Generalizable Gaze Estimation
- Authors: Yihua Cheng, Yiwei Bao, Feng Lu
- Abstract summary: We tackle the domain generalization problem in cross-domain gaze estimation for unknown target domains.
To be specific, we realize the domain generalization by gaze feature purification.
We design a plug-and-play self-adversarial framework for the gaze feature purification.
- Score: 12.076469954457007
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Gaze estimation methods learn eye gaze from facial features. However, among
rich information in the facial image, real gaze-relevant features only
correspond to subtle changes in eye region, while other gaze-irrelevant
features like illumination, personal appearance and even facial expression may
affect the learning in an unexpected way. This is a major reason why existing
methods show significant performance degradation in cross-domain/dataset
evaluation. In this paper, we tackle the domain generalization problem in
cross-domain gaze estimation for unknown target domains. To be specific, we
realize the domain generalization by gaze feature purification. We eliminate
gaze-irrelevant factors such as illumination and identity to improve the
cross-dataset performance without knowing the target dataset. We design a
plug-and-play self-adversarial framework for the gaze feature purification. The
framework enhances not only our baseline but also existing gaze estimation
methods directly and significantly. Our method achieves the state-of-the-art
performance in different benchmarks. Meanwhile, the purification is easily
explainable via visualization.
Related papers
- CLIP-Gaze: Towards General Gaze Estimation via Visual-Linguistic Model [13.890404285565225]
We propose a novel framework called CLIP-Gaze that utilizes a pre-trained vision-language model to leverage its transferable knowledge.
Our framework is the first to leverage the vision-and-language cross-modality approach for gaze estimation task.
arXiv Detail & Related papers (2024-03-08T07:37:21Z) - Keypoint Description by Symmetry Assessment -- Applications in
Biometrics [49.547569925407814]
We present a model-based feature extractor to describe neighborhoods around keypoints by finite expansion.
The iso-curves of such functions are highly symmetric w.r.t. the origin (a keypoint) and the estimated parameters have well defined geometric interpretations.
arXiv Detail & Related papers (2023-11-03T00:49:25Z) - Bridging the Gap: Gaze Events as Interpretable Concepts to Explain Deep
Neural Sequence Models [0.7829352305480283]
In this work, we employ established gaze event detection algorithms for fixations and saccades.
We quantitatively evaluate the impact of these events by determining their concept influence.
arXiv Detail & Related papers (2023-04-12T10:15:31Z) - Cluster-level pseudo-labelling for source-free cross-domain facial
expression recognition [94.56304526014875]
We propose the first Source-Free Unsupervised Domain Adaptation (SFUDA) method for Facial Expression Recognition (FER)
Our method exploits self-supervised pretraining to learn good feature representations from the target data.
We validate the effectiveness of our method in four adaptation setups, proving that it consistently outperforms existing SFUDA methods when applied to FER.
arXiv Detail & Related papers (2022-10-11T08:24:50Z) - Jitter Does Matter: Adapting Gaze Estimation to New Domains [12.482427155726413]
We propose to utilize gaze jitter to analyze and optimize gaze domain adaptation task.
We find that the high-frequency component (HFC) is an important factor that leads to jitter.
We employ contrastive learning to encourage the model to obtain similar representations between original and perturbed data.
arXiv Detail & Related papers (2022-10-05T08:20:41Z) - LatentGaze: Cross-Domain Gaze Estimation through Gaze-Aware Analytic
Latent Code Manipulation [0.0]
We propose a gaze-aware analytic manipulation method, based on a data-driven approach with generative adversarial network inversion's disentanglement characteristics.
By utilizing GAN-based encoder-generator process, we shift the input image from the target domain to the source domain image, which a gaze estimator is sufficiently aware.
arXiv Detail & Related papers (2022-09-21T08:05:53Z) - Active Gaze Control for Foveal Scene Exploration [124.11737060344052]
We propose a methodology to emulate how humans and robots with foveal cameras would explore a scene.
The proposed method achieves an increase in detection F1-score of 2-3 percentage points for the same number of gaze shifts.
arXiv Detail & Related papers (2022-08-24T14:59:28Z) - Causal Transportability for Visual Recognition [70.13627281087325]
We show that standard classifiers fail because the association between images and labels is not transportable across settings.
We then show that the causal effect, which severs all sources of confounding, remains invariant across domains.
This motivates us to develop an algorithm to estimate the causal effect for image classification.
arXiv Detail & Related papers (2022-04-26T15:02:11Z) - Goal-Oriented Gaze Estimation for Zero-Shot Learning [62.52340838817908]
We introduce a novel goal-oriented gaze estimation module (GEM) to improve the discriminative attribute localization.
We aim to predict the actual human gaze location to get the visual attention regions for recognizing a novel object guided by attribute description.
This work implies the promising benefits of collecting human gaze dataset and automatic gaze estimation algorithms on high-level computer vision tasks.
arXiv Detail & Related papers (2021-03-05T02:14:57Z) - Visualization of Supervised and Self-Supervised Neural Networks via
Attribution Guided Factorization [87.96102461221415]
We develop an algorithm that provides per-class explainability.
In an extensive battery of experiments, we demonstrate the ability of our methods to class-specific visualization.
arXiv Detail & Related papers (2020-12-03T18:48:39Z) - Integrating Human Gaze into Attention for Egocentric Activity
Recognition [40.517438760096056]
We introduce an effective probabilistic approach to integrate human gaze intotemporal attention for egocentric activity recognition.
We represent the locations gaze fixation points as structured discrete latent variables to model their uncertainties.
The predicted gaze locations are used to provide informative attentional cues to improve the recognition performance.
arXiv Detail & Related papers (2020-11-08T08:02:30Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.