FreeGaze: Resource-efficient Gaze Estimation via Frequency Domain Contrastive Learning

• URL: http://arxiv.org/abs/2209.06692v1
• Date: Wed, 14 Sep 2022 14:51:52 GMT
• Title: FreeGaze: Resource-efficient Gaze Estimation via Frequency Domain Contrastive Learning
• Authors: Lingyu Du, Guohao Lan
• Abstract summary: FreeGaze is a resource-efficient framework for unsupervised gaze representation learning. We show that FreeGaze can achieve comparable gaze estimation accuracy with existing supervised learning-based approach.
• Score: 1.240096657086732
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Gaze estimation is of great importance to many scientific fields and daily applications, ranging from fundamental research in cognitive psychology to attention-aware mobile systems. While recent advancements in deep learning have yielded remarkable successes in building highly accurate gaze estimation systems, the associated high computational cost and the reliance on large-scale labeled gaze data for supervised learning place challenges on the practical use of existing solutions. To move beyond these limitations, we present FreeGaze, a resource-efficient framework for unsupervised gaze representation learning. FreeGaze incorporates the frequency domain gaze estimation and the contrastive gaze representation learning in its design. The former significantly alleviates the computational burden in both system calibration and gaze estimation, and dramatically reduces the system latency; while the latter overcomes the data labeling hurdle of existing supervised learning-based counterparts, and ensures efficient gaze representation learning in the absence of gaze label. Our evaluation on two gaze estimation datasets shows that FreeGaze can achieve comparable gaze estimation accuracy with existing supervised learning-based approach, while enabling up to 6.81 and 1.67 times speedup in system calibration and gaze estimation, respectively.

Related papers

• UniGaze: Towards Universal Gaze Estimation via Large-scale Pre-Training [12.680014448486242]
  We propose UniGaze, leveraging large-scale in-the-wild facial datasets for gaze estimation through self-supervised pre-training. Our experiments reveal that self-supervised approaches designed for semantic tasks fail when applied to gaze estimation. We demonstrate that UniGaze significantly improves generalization across multiple data domains while minimizing reliance on costly labeled data.
  arXiv  Detail & Related papers  (2025-02-04T13:24:23Z)
• Modeling State Shifting via Local-Global Distillation for Event-Frame Gaze Tracking [61.44701715285463]
  This paper tackles the problem of passive gaze estimation using both event and frame data. We reformulate gaze estimation as the quantification of the state shifting from the current state to several prior registered anchor states. To improve the generalization ability, instead of learning a large gaze estimation network directly, we align a group of local experts with a student network.
  arXiv  Detail & Related papers  (2024-03-31T03:30:37Z)
• Overcoming Pitfalls in Graph Contrastive Learning Evaluation: Toward Comprehensive Benchmarks [60.82579717007963]
  We introduce an enhanced evaluation framework designed to more accurately gauge the effectiveness, consistency, and overall capability of Graph Contrastive Learning (GCL) methods.
  arXiv  Detail & Related papers  (2024-02-24T01:47:56Z)
• Unsupervised Gaze-aware Contrastive Learning with Subject-specific Condition [6.547550920819356]
  ConGaze is a contrastive learning-based framework that learns generic gaze-aware representations across subjects in an unsupervised way. We introduce gaze-specific data augmentation to preserve the gaze-semantic features and maintain the gaze consistency. We also devise a novel subject-conditional projection module that encourages a share feature extractor to learn gaze-aware and generic representations.
  arXiv  Detail & Related papers  (2023-09-08T09:45:19Z)
• Fairness Improves Learning from Noisily Labeled Long-Tailed Data [119.0612617460727]
  Long-tailed and noisily labeled data frequently appear in real-world applications and impose significant challenges for learning. We introduce the Fairness Regularizer (FR), inspired by regularizing the performance gap between any two sub-populations. We show that the introduced fairness regularizer improves the performances of sub-populations on the tail and the overall learning performance.
  arXiv  Detail & Related papers  (2023-03-22T03:46:51Z)
• LatentGaze: Cross-Domain Gaze Estimation through Gaze-Aware Analytic Latent Code Manipulation [0.0]
  We propose a gaze-aware analytic manipulation method, based on a data-driven approach with generative adversarial network inversion's disentanglement characteristics. By utilizing GAN-based encoder-generator process, we shift the input image from the target domain to the source domain image, which a gaze estimator is sufficiently aware.
  arXiv  Detail & Related papers  (2022-09-21T08:05:53Z)
• Active Gaze Control for Foveal Scene Exploration [124.11737060344052]
  We propose a methodology to emulate how humans and robots with foveal cameras would explore a scene. The proposed method achieves an increase in detection F1-score of 2-3 percentage points for the same number of gaze shifts.
  arXiv  Detail & Related papers  (2022-08-24T14:59:28Z)
• Challenges and Opportunities in Offline Reinforcement Learning from Visual Observations [58.758928936316785]
  offline reinforcement learning from visual observations with continuous action spaces remains under-explored. We show that modifications to two popular vision-based online reinforcement learning algorithms suffice to outperform existing offline RL methods.
  arXiv  Detail & Related papers  (2022-06-09T22:08:47Z)
• Effect Of Personalized Calibration On Gaze Estimation Using Deep-Learning [10.815594142396497]
  We train a convolutional neural network and analyse its performance with and without calibration. This evaluation provides clear insights on how calibration improved the performance of the Deep Learning model in estimating gaze in the wild.
  arXiv  Detail & Related papers  (2021-09-27T05:14:12Z)
• Weakly-Supervised Physically Unconstrained Gaze Estimation [80.66438763587904]
  We tackle the previously unexplored problem of weakly-supervised gaze estimation from videos of human interactions. We propose a training algorithm along with several novel loss functions especially designed for the task. We show significant improvements in (a) the accuracy of semi-supervised gaze estimation and (b) cross-domain generalization on the state-of-the-art physically unconstrained in-the-wild Gaze360 gaze estimation benchmark.
  arXiv  Detail & Related papers  (2021-05-20T14:58:52Z)
• Appearance-based Gaze Estimation With Deep Learning: A Review and Benchmark [14.306488668615883]
  We present a systematic review of the appearance-based gaze estimation methods using deep learning. We summarize the data pre-processing and post-processing methods, including face/eye detection, data rectification, 2D/3D gaze conversion and gaze origin conversion.
  arXiv  Detail & Related papers  (2021-04-26T15:53:03Z)
• Integrating Human Gaze into Attention for Egocentric Activity Recognition [40.517438760096056]
  We introduce an effective probabilistic approach to integrate human gaze intotemporal attention for egocentric activity recognition. We represent the locations gaze fixation points as structured discrete latent variables to model their uncertainties. The predicted gaze locations are used to provide informative attentional cues to improve the recognition performance.
  arXiv  Detail & Related papers  (2020-11-08T08:02:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.