Precise Facial Landmark Detection by Reference Heatmap Transformer

• URL: http://arxiv.org/abs/2303.07840v1
• Date: Tue, 14 Mar 2023 12:26:48 GMT
• Title: Precise Facial Landmark Detection by Reference Heatmap Transformer
• Authors: Jun Wan, Jun Liu, Jie Zhou, Zhihui Lai, Linlin Shen, Hang Sun, Ping Xiong, Wenwen Min
• Abstract summary: We propose a novel Reference Heatmap Transformer (RHT) for more precise facial landmark detection. The experimental results from challenging benchmark datasets demonstrate that our proposed method outperforms the state-of-the-art methods in the literature.
• Score: 52.417964103227696
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Most facial landmark detection methods predict landmarks by mapping the input facial appearance features to landmark heatmaps and have achieved promising results. However, when the face image is suffering from large poses, heavy occlusions and complicated illuminations, they cannot learn discriminative feature representations and effective facial shape constraints, nor can they accurately predict the value of each element in the landmark heatmap, limiting their detection accuracy. To address this problem, we propose a novel Reference Heatmap Transformer (RHT) by introducing reference heatmap information for more precise facial landmark detection. The proposed RHT consists of a Soft Transformation Module (STM) and a Hard Transformation Module (HTM), which can cooperate with each other to encourage the accurate transformation of the reference heatmap information and facial shape constraints. Then, a Multi-Scale Feature Fusion Module (MSFFM) is proposed to fuse the transformed heatmap features and the semantic features learned from the original face images to enhance feature representations for producing more accurate target heatmaps. To the best of our knowledge, this is the first study to explore how to enhance facial landmark detection by transforming the reference heatmap information. The experimental results from challenging benchmark datasets demonstrate that our proposed method outperforms the state-of-the-art methods in the literature.

Related papers

• VcT: Visual change Transformer for Remote Sensing Image Change Detection [16.778418602705287]
  We propose a novel Visual change Transformer (VcT) model for visual change detection problem. Top-K reliable tokens can be mined from the map and refined by using the clustering algorithm. Extensive experiments on multiple benchmark datasets validated the effectiveness of our proposed VcT model.
  arXiv  Detail & Related papers  (2023-10-17T17:25:31Z)
• Subpixel Heatmap Regression for Facial Landmark Localization [65.41270740933656]
  Heatmap regression approaches suffer from discretization-induced errors related to both the heatmap encoding and decoding process. We propose a new approach for the heatmap encoding and decoding process by leveraging the underlying continuous distribution. Our approach offers noticeable gains across multiple datasets setting a new state-of-the-art result in facial landmark localization.
  arXiv  Detail & Related papers  (2021-11-03T17:21:28Z)
• Robust Face Alignment by Multi-order High-precision Hourglass Network [44.94500006611075]
  This paper proposes a heatmap subpixel regression (HSR) method and a multi-order cross geometry-aware (MCG) model. The HSR method is proposed to achieve high-precision landmark detection by a well-designed subpixel detection loss (SDL) and subpixel detection technology (SDT) At the same time, the MCG model is able to use the proposed multi-order cross information to learn more discriminative representations for enhancing facial geometric constraints and context information.
  arXiv  Detail & Related papers  (2020-10-17T05:40:30Z)
• Heatmap Regression via Randomized Rounding [105.75014893647538]
  We propose a simple yet effective quantization system to address the sub-pixel localization problem. The proposed system encodes the fractional part of numerical coordinates into the ground truth heatmap using a probabilistic approach during training.
  arXiv  Detail & Related papers  (2020-09-01T04:54:22Z)
• Multi-spectral Facial Landmark Detection [10.009879315990133]
  We propose a robust neural network enabled facial landmark detection, namely Deep Multi-Spectral Learning (DMSL) DMSL consists of two sub-models, i.e. face boundary detection, and landmark coordinates detection. Experiment conducted on Eurecom's visible and thermal paired database shows the superior performance of DMSL over the state-of-the-art for thermal facial landmark detection.
  arXiv  Detail & Related papers  (2020-06-09T11:43:46Z)
• Multi-Scale Thermal to Visible Face Verification via Attribute Guided Synthesis [55.29770222566124]
  We use attributes extracted from visible images to synthesize attribute-preserved visible images from thermal imagery for cross-modal matching. A novel multi-scale generator is proposed to synthesize the visible image from the thermal image guided by the extracted attributes. A pre-trained VGG-Face network is leveraged to extract features from the synthesized image and the input visible image for verification.
  arXiv  Detail & Related papers  (2020-04-20T01:45:05Z)
• A Transfer Learning approach to Heatmap Regression for Action Unit intensity estimation [50.261472059743845]
  Action Units (AUs) are geometrically-based atomic facial muscle movements. We propose a novel AU modelling problem that consists of jointly estimating their localisation and intensity. A Heatmap models whether an AU occurs or not at a given spatial location.
  arXiv  Detail & Related papers  (2020-04-14T16:51:13Z)
• Attentive One-Dimensional Heatmap Regression for Facial Landmark Detection and Tracking [73.35078496883125]
  We propose a novel attentive one-dimensional heatmap regression method for facial landmark localization. First, we predict two groups of 1D heatmaps to represent the marginal distributions of the x and y coordinates. Second, a co-attention mechanism is adopted to model the inherent spatial patterns existing in x and y coordinates. Third, based on the 1D heatmap structures, we propose a facial landmark detector capturing spatial patterns for landmark detection on an image.
  arXiv  Detail & Related papers  (2020-04-05T06:51:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.