Precise Facial Landmark Detection by Reference Heatmap Transformer
- URL: http://arxiv.org/abs/2303.07840v1
- Date: Tue, 14 Mar 2023 12:26:48 GMT
- Title: Precise Facial Landmark Detection by Reference Heatmap Transformer
- Authors: Jun Wan, Jun Liu, Jie Zhou, Zhihui Lai, Linlin Shen, Hang Sun, Ping
Xiong, Wenwen Min
- Abstract summary: We propose a novel Reference Heatmap Transformer (RHT) for more precise facial landmark detection.
The experimental results from challenging benchmark datasets demonstrate that our proposed method outperforms the state-of-the-art methods in the literature.
- Score: 52.417964103227696
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Most facial landmark detection methods predict landmarks by mapping the input
facial appearance features to landmark heatmaps and have achieved promising
results. However, when the face image is suffering from large poses, heavy
occlusions and complicated illuminations, they cannot learn discriminative
feature representations and effective facial shape constraints, nor can they
accurately predict the value of each element in the landmark heatmap, limiting
their detection accuracy. To address this problem, we propose a novel Reference
Heatmap Transformer (RHT) by introducing reference heatmap information for more
precise facial landmark detection. The proposed RHT consists of a Soft
Transformation Module (STM) and a Hard Transformation Module (HTM), which can
cooperate with each other to encourage the accurate transformation of the
reference heatmap information and facial shape constraints. Then, a Multi-Scale
Feature Fusion Module (MSFFM) is proposed to fuse the transformed heatmap
features and the semantic features learned from the original face images to
enhance feature representations for producing more accurate target heatmaps. To
the best of our knowledge, this is the first study to explore how to enhance
facial landmark detection by transforming the reference heatmap information.
The experimental results from challenging benchmark datasets demonstrate that
our proposed method outperforms the state-of-the-art methods in the literature.
Related papers
- VcT: Visual change Transformer for Remote Sensing Image Change Detection [16.778418602705287]
We propose a novel Visual change Transformer (VcT) model for visual change detection problem.
Top-K reliable tokens can be mined from the map and refined by using the clustering algorithm.
Extensive experiments on multiple benchmark datasets validated the effectiveness of our proposed VcT model.
arXiv Detail & Related papers (2023-10-17T17:25:31Z) - Subpixel Heatmap Regression for Facial Landmark Localization [65.41270740933656]
Heatmap regression approaches suffer from discretization-induced errors related to both the heatmap encoding and decoding process.
We propose a new approach for the heatmap encoding and decoding process by leveraging the underlying continuous distribution.
Our approach offers noticeable gains across multiple datasets setting a new state-of-the-art result in facial landmark localization.
arXiv Detail & Related papers (2021-11-03T17:21:28Z) - Robust Face Alignment by Multi-order High-precision Hourglass Network [44.94500006611075]
This paper proposes a heatmap subpixel regression (HSR) method and a multi-order cross geometry-aware (MCG) model.
The HSR method is proposed to achieve high-precision landmark detection by a well-designed subpixel detection loss (SDL) and subpixel detection technology (SDT)
At the same time, the MCG model is able to use the proposed multi-order cross information to learn more discriminative representations for enhancing facial geometric constraints and context information.
arXiv Detail & Related papers (2020-10-17T05:40:30Z) - Heatmap Regression via Randomized Rounding [105.75014893647538]
We propose a simple yet effective quantization system to address the sub-pixel localization problem.
The proposed system encodes the fractional part of numerical coordinates into the ground truth heatmap using a probabilistic approach during training.
arXiv Detail & Related papers (2020-09-01T04:54:22Z) - Multi-spectral Facial Landmark Detection [10.009879315990133]
We propose a robust neural network enabled facial landmark detection, namely Deep Multi-Spectral Learning (DMSL)
DMSL consists of two sub-models, i.e. face boundary detection, and landmark coordinates detection.
Experiment conducted on Eurecom's visible and thermal paired database shows the superior performance of DMSL over the state-of-the-art for thermal facial landmark detection.
arXiv Detail & Related papers (2020-06-09T11:43:46Z) - Multi-Scale Thermal to Visible Face Verification via Attribute Guided
Synthesis [55.29770222566124]
We use attributes extracted from visible images to synthesize attribute-preserved visible images from thermal imagery for cross-modal matching.
A novel multi-scale generator is proposed to synthesize the visible image from the thermal image guided by the extracted attributes.
A pre-trained VGG-Face network is leveraged to extract features from the synthesized image and the input visible image for verification.
arXiv Detail & Related papers (2020-04-20T01:45:05Z) - A Transfer Learning approach to Heatmap Regression for Action Unit
intensity estimation [50.261472059743845]
Action Units (AUs) are geometrically-based atomic facial muscle movements.
We propose a novel AU modelling problem that consists of jointly estimating their localisation and intensity.
A Heatmap models whether an AU occurs or not at a given spatial location.
arXiv Detail & Related papers (2020-04-14T16:51:13Z) - Attentive One-Dimensional Heatmap Regression for Facial Landmark
Detection and Tracking [73.35078496883125]
We propose a novel attentive one-dimensional heatmap regression method for facial landmark localization.
First, we predict two groups of 1D heatmaps to represent the marginal distributions of the x and y coordinates.
Second, a co-attention mechanism is adopted to model the inherent spatial patterns existing in x and y coordinates.
Third, based on the 1D heatmap structures, we propose a facial landmark detector capturing spatial patterns for landmark detection on an image.
arXiv Detail & Related papers (2020-04-05T06:51:22Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.