Deep Entwined Learning Head Pose and Face Alignment Inside an Attentional Cascade with Doubly-Conditional fusion

• URL: http://arxiv.org/abs/2004.06558v1
• Date: Tue, 14 Apr 2020 14:42:35 GMT
• Title: Deep Entwined Learning Head Pose and Face Alignment Inside an Attentional Cascade with Doubly-Conditional fusion
• Authors: Arnaud Dapogny, K\'evin Bailly and Matthieu Cord
• Abstract summary: Head pose estimation and face alignment constitute a backbone preprocessing for many applications relying on face analysis. We propose to entwine face alignment and head pose tasks inside an attentional cascade. We empirically show the benefit of entwining head pose and landmark localization objectives inside our architecture.
• Score: 42.50876580245864
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Head pose estimation and face alignment constitute a backbone preprocessing for many applications relying on face analysis. While both are closely related tasks, they are generally addressed separately, e.g. by deducing the head pose from the landmark locations. In this paper, we propose to entwine face alignment and head pose tasks inside an attentional cascade. This cascade uses a geometry transfer network for integrating heterogeneous annotations to enhance landmark localization accuracy. Furthermore, we propose a doubly-conditional fusion scheme to select relevant feature maps, and regions thereof, based on a current head pose and landmark localization estimate. We empirically show the benefit of entwining head pose and landmark localization objectives inside our architecture, and that the proposed AC-DC model enhances the state-of-the-art accuracy on multiple databases for both face alignment and head pose estimation tasks.

Related papers

• 6DoF Head Pose Estimation through Explicit Bidirectional Interaction with Face Geometry [3.106167803320563]
  This study addresses the nuanced challenge of estimating head translations within the context of 6DoF head pose estimation. We propose a novel approach called the head Translation, Rotation, and face Geometry network (TRG), which stands out for its explicit bidirectional interaction structure. Our contributions also include the development of a strategy for estimating bounding box correction parameters and a technique for aligning landmarks to image.
  arXiv  Detail & Related papers  (2024-07-19T09:05:49Z)
• HS-Diffusion: Semantic-Mixing Diffusion for Head Swapping [150.06405071177048]
  We propose a semantic-mixing diffusion model for head swapping (HS-Diffusion) We blend the semantic layouts of source head and source body, and then inpaint the transition region by the semantic layout generator. We construct a new image-based head swapping benchmark and design two tailor-designed metrics.
  arXiv  Detail & Related papers  (2022-12-13T10:04:01Z)
• Shape Preserving Facial Landmarks with Graph Attention Networks [3.996275177789895]
  We propose a model based on the combination of a CNN with a cascade of Graph Attention Network regressors. We introduce an encoding that jointly represents the appearance and location of facial landmarks and an attention mechanism to weigh the information according to its reliability. Experiments confirm that the proposed model learns a global representation of the structure of the face, achieving top performance in popular benchmarks on head pose and landmark estimation.
  arXiv  Detail & Related papers  (2022-10-13T17:58:02Z)
• Towards Accurate Facial Landmark Detection via Cascaded Transformers [14.74021483826222]
  We propose an accurate facial landmark detector based on cascaded transformers. With self-attention in transformers, our model can inherently exploit the structured relationships between landmarks. During cascaded refinement, our model is able to extract the most relevant image features around the target landmark for coordinate prediction.
  arXiv  Detail & Related papers  (2022-08-23T08:42:13Z)
• RePFormer: Refinement Pyramid Transformer for Robust Facial Landmark Detection [131.1478251760399]
  We formulate the facial landmark detection task as refining landmark queries along pyramid memories. Specifically, a pyramid transformer head (PTH) is introduced to build both relations among landmarks and heterologous relations between landmarks and cross-scale contexts. A dynamic landmark refinement (DLR) module is designed to decompose the landmark regression into an end-to-end refinement procedure.
  arXiv  Detail & Related papers  (2022-07-08T14:12:26Z)
• Towards Self-Supervised Category-Level Object Pose and Size Estimation [121.28537953301951]
  This work presents a self-supervised framework for category-level object pose and size estimation from a single depth image. We leverage the geometric consistency residing in point clouds of the same shape for self-supervision.
  arXiv  Detail & Related papers  (2022-03-06T06:02:30Z)
• Multi-task head pose estimation in-the-wild [7.476901945542385]
  We present a deep learning-based multi-task approach for head pose estimation in images. We harness the strong dependencies among face pose, alignment and visibility to produce a top performing model for all three tasks.
  arXiv  Detail & Related papers  (2022-02-04T18:35:52Z)
• SIRI: Spatial Relation Induced Network For Spatial Description Resolution [64.38872296406211]
  We propose a novel relationship induced (SIRI) network for language-guided localization. We show that our method is around 24% better than the state-of-the-art method in terms of accuracy, measured by an 80-pixel radius. Our method also generalizes well on our proposed extended dataset collected using the same settings as Touchdown.
  arXiv  Detail & Related papers  (2020-10-27T14:04:05Z)
• Deep Structured Prediction for Facial Landmark Detection [59.60946775628646]
  This paper proposes a method for deep structured facial landmark detection based on combining a deep Convolutional Network with a Conditional Random Field. We demonstrate its superior performance to existing state-of-the-art techniques in facial landmark detection.
  arXiv  Detail & Related papers  (2020-10-18T17:09:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.