Reasoning Structural Relation for Occlusion-Robust Facial Landmark Localization

• URL: http://arxiv.org/abs/2112.10087v1
• Date: Sun, 19 Dec 2021 08:27:37 GMT
• Title: Reasoning Structural Relation for Occlusion-Robust Facial Landmark Localization
• Authors: Congcong Zhu, Xiaoqiang Li, Jide Li, Songmin Dai, Weiqin Tong
• Abstract summary: This paper proposes a structural relation network (SRN) for occlusion-robust landmark localization. Unlike most existing methods that simply exploit the shape constraint, the proposed SRN aims to capture the structural relations among different facial components. The proposed method achieves outstanding performance on occluded and masked faces.
• Score: 5.171058506312429
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In facial landmark localization tasks, various occlusions heavily degrade the localization accuracy due to the partial observability of facial features. This paper proposes a structural relation network (SRN) for occlusion-robust landmark localization. Unlike most existing methods that simply exploit the shape constraint, the proposed SRN aims to capture the structural relations among different facial components. These relations can be considered a more powerful shape constraint against occlusion. To achieve this, a hierarchical structural relation module (HSRM) is designed to hierarchically reason the structural relations that represent both long- and short-distance spatial dependencies. Compared with existing network architectures, HSRM can efficiently model the spatial relations by leveraging its geometry-aware network architecture, which reduces the semantic ambiguity caused by occlusion. Moreover, the SRN augments the training data by synthesizing occluded faces. To further extend our SRN for occluded video data, we formulate the occluded face synthesis as a Markov decision process (MDP). Specifically, it plans the movement of the dynamic occlusion based on an accumulated reward associated with the performance degradation of the pre-trained SRN. This procedure augments hard samples for robust facial landmark tracking. Extensive experimental results indicate that the proposed method achieves outstanding performance on occluded and masked faces. Code is available at https://github.com/zhuccly/SRN.

Related papers

• Robust Ensemble Person Re-Identification via Orthogonal Fusion with Occlusion Handling [4.431087385310259]
  Occlusion remains one of the major challenges in person reidentification (ReID) We propose a deep ensemble model that harnesses both CNN and Transformer architectures to generate robust feature representations.
  arXiv  Detail & Related papers  (2024-03-29T18:38:59Z)
• Rotated Multi-Scale Interaction Network for Referring Remote Sensing Image Segmentation [63.15257949821558]
  Referring Remote Sensing Image (RRSIS) is a new challenge that combines computer vision and natural language processing. Traditional Referring Image (RIS) approaches have been impeded by the complex spatial scales and orientations found in aerial imagery. We introduce the Rotated Multi-Scale Interaction Network (RMSIN), an innovative approach designed for the unique demands of RRSIS.
  arXiv  Detail & Related papers  (2023-12-19T08:14:14Z)
• Iterative Soft Shrinkage Learning for Efficient Image Super-Resolution [91.3781512926942]
  Image super-resolution (SR) has witnessed extensive neural network designs from CNN to transformer architectures. This work investigates the potential of network pruning for super-resolution iteration to take advantage of off-the-shelf network designs and reduce the underlying computational overhead. We propose a novel Iterative Soft Shrinkage-Percentage (ISS-P) method by optimizing the sparse structure of a randomly network at each and tweaking unimportant weights with a small amount proportional to the magnitude scale on-the-fly.
  arXiv  Detail & Related papers  (2023-03-16T21:06:13Z)
• Controllable and Guided Face Synthesis for Unconstrained Face Recognition [17.08390901848988]
  We propose a controllable face synthesis model (CFSM) that can mimic the distribution of target datasets in a style latent space. CFSM learns a linear subspace with orthogonal bases in the style latent space with precise control over the diversity and degree of synthesis. Our approach yields significant performance gains on unconstrained benchmarks, such as IJB-B, IJB-C, TinyFace and IJB-S.
  arXiv  Detail & Related papers  (2022-07-20T20:13:29Z)
• Semi-signed neural fitting for surface reconstruction from unoriented point clouds [53.379712818791894]
  We propose SSN-Fitting to reconstruct a better signed distance field. SSN-Fitting consists of a semi-signed supervision and a loss-based region sampling strategy. We conduct experiments to demonstrate that SSN-Fitting achieves state-of-the-art performance under different settings.
  arXiv  Detail & Related papers  (2022-06-14T09:40:17Z)
• Decoupled Multi-task Learning with Cyclical Self-Regulation for Face Parsing [71.19528222206088]
  We propose a novel Decoupled Multi-task Learning with Cyclical Self-Regulation for face parsing. Specifically, DML-CSR designs a multi-task model which comprises face parsing, binary edge, and category edge detection. Our method achieves the new state-of-the-art performance on the Helen, CelebA-HQ, and LapaMask datasets.
  arXiv  Detail & Related papers  (2022-03-28T02:12:30Z)
• Region attention and graph embedding network for occlusion objective class-based micro-expression recognition [26.5638344747854]
  Micro-expression recognition (textbfMER) has attracted lots of researchers' attention in a decade. This paper deeply investigates an interesting but unexplored challenging issue in MER, ie, occlusion MER. A underlineRegion-inspired underlineRelation underlineReasoning underlineNetwork (textbfRRRN) is proposed to model relations between various facial regions.
  arXiv  Detail & Related papers  (2021-07-13T08:04:03Z)
• Multi-Margin based Decorrelation Learning for Heterogeneous Face Recognition [90.26023388850771]
  This paper presents a deep neural network approach to extract decorrelation representations in a hyperspherical space for cross-domain face images. The proposed framework can be divided into two components: heterogeneous representation network and decorrelation representation learning. Experimental results on two challenging heterogeneous face databases show that our approach achieves superior performance on both verification and recognition tasks.
  arXiv  Detail & Related papers  (2020-05-25T07:01:12Z)
• Relational Deep Feature Learning for Heterogeneous Face Recognition [17.494718795454055]
  We propose a graph-structured module called Graph Module (NIR) that extracts global relational information in addition to general facial features. The proposed method outperforms other state-of-the-art methods on five Heterogeneous Face Recognition (HFR) databases.
  arXiv  Detail & Related papers  (2020-03-02T07:35:23Z)
• SD-GAN: Structural and Denoising GAN reveals facial parts under occlusion [7.284661356980246]
  We propose a generative model to reconstruct the missing parts of the face which are under occlusion. A novel adversarial training algorithm has been designed for a bimodal mutually exclusive Generative Adversarial Network (GAN) model. Our proposed technique outperforms the competing methods by a considerable margin, even for boosting the performance of Face Recognition.
  arXiv  Detail & Related papers  (2020-02-19T21:12:49Z)
• Local Propagation in Constraint-based Neural Network [77.37829055999238]
  We study a constraint-based representation of neural network architectures. We investigate a simple optimization procedure that is well suited to fulfil the so-called architectural constraints.
  arXiv  Detail & Related papers  (2020-02-18T16:47:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.