Precise Facial Landmark Detection by Dynamic Semantic Aggregation Transformer

• URL: http://arxiv.org/abs/2412.00740v1
• Date: Sun, 01 Dec 2024 09:20:32 GMT
• Title: Precise Facial Landmark Detection by Dynamic Semantic Aggregation Transformer
• Authors: Jun Wan, He Liu, Yujia Wu, Zhihui Lai, Wenwen Min, Jun Liu,
• Abstract summary: Deep neural network methods have played a dominant role in face alignment field. We propose a Dynamic Semantic-Aggregation Transformer (DSAT) for more discriminative and representative feature learning. Our proposed DSAT outperforms state-of-the-art models in the literature.
• Score: 29.484887366344363
• License:
• Abstract: At present, deep neural network methods have played a dominant role in face alignment field. However, they generally use predefined network structures to predict landmarks, which tends to learn general features and leads to mediocre performance, e.g., they perform well on neutral samples but struggle with faces exhibiting large poses or occlusions. Moreover, they cannot effectively deal with semantic gaps and ambiguities among features at different scales, which may hinder them from learning efficient features. To address the above issues, in this paper, we propose a Dynamic Semantic-Aggregation Transformer (DSAT) for more discriminative and representative feature (i.e., specialized feature) learning. Specifically, a Dynamic Semantic-Aware (DSA) model is first proposed to partition samples into subsets and activate the specific pathways for them by estimating the semantic correlations of feature channels, making it possible to learn specialized features from each subset. Then, a novel Dynamic Semantic Specialization (DSS) model is designed to mine the homogeneous information from features at different scales for eliminating the semantic gap and ambiguities and enhancing the representation ability. Finally, by integrating the DSA model and DSS model into our proposed DSAT in both dynamic architecture and dynamic parameter manners, more specialized features can be learned for achieving more precise face alignment. It is interesting to show that harder samples can be handled by activating more feature channels. Extensive experiments on popular face alignment datasets demonstrate that our proposed DSAT outperforms state-of-the-art models in the literature.Our code is available at https://github.com/GERMINO-LiuHe/DSAT.

Related papers

• How to Squeeze An Explanation Out of Your Model [13.154512864498912]
  This paper proposes an approach for interpretability that is model-agnostic. By including an SE block prior to the classification layer of any model, we are able to retrieve the most influential features. Results show that this new SE-based interpretability can be applied to various models in image and video/multi-modal settings.
  arXiv  Detail & Related papers  (2024-12-06T15:47:53Z)
• Dynamical similarity analysis can identify compositional dynamics developing in RNNs [3.037387520023979]
  compositional learning in neural networks (RNNs) allows us to build a test case for dynamical representation alignment metrics. We show that the new Dynamical Similarity Analysis (DSA) is more noise robust and identifies behaviorally relevant representations more reliably than prior metrics.
  arXiv  Detail & Related papers  (2024-10-31T16:07:21Z)
• Generalizable Implicit Neural Representation As a Universal Spatiotemporal Traffic Data Learner [46.866240648471894]
  Spatiotemporal Traffic Data (STTD) measures the complex dynamical behaviors of the multiscale transportation system. We present a novel paradigm to address the STTD learning problem by parameterizing STTD as an implicit neural representation. We validate its effectiveness through extensive experiments in real-world scenarios, showcasing applications from corridor to network scales.
  arXiv  Detail & Related papers  (2024-06-13T02:03:22Z)
• Image Captioning via Dynamic Path Customization [100.15412641586525]
  We propose a novel Dynamic Transformer Network (DTNet) for image captioning, which dynamically assigns customized paths to different samples, leading to discriminative yet accurate captions. To validate the effectiveness of our proposed DTNet, we conduct extensive experiments on the MS-COCO dataset and achieve new state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-06-01T07:23:21Z)
• Spatiotemporal Implicit Neural Representation as a Generalized Traffic Data Learner [46.866240648471894]
  Spatiotemporal Traffic Data (STTD) measures the complex dynamical behaviors of the multiscale transportation system. We present a novel paradigm to address the STTD learning problem by parameterizing STTD as an implicit neural representation. We validate its effectiveness through extensive experiments in real-world scenarios, showcasing applications from corridor to network scales.
  arXiv  Detail & Related papers  (2024-05-06T06:23:06Z)
• Prompt-Driven Dynamic Object-Centric Learning for Single Domain Generalization [61.64304227831361]
  Single-domain generalization aims to learn a model from single source domain data to achieve generalized performance on other unseen target domains. We propose a dynamic object-centric perception network based on prompt learning, aiming to adapt to the variations in image complexity.
  arXiv  Detail & Related papers  (2024-02-28T16:16:51Z)
• ContraFeat: Contrasting Deep Features for Semantic Discovery [102.4163768995288]
  StyleGAN has shown strong potential for disentangled semantic control. Existing semantic discovery methods on StyleGAN rely on manual selection of modified latent layers to obtain satisfactory manipulation results. We propose a model that automates this process and achieves state-of-the-art semantic discovery performance.
  arXiv  Detail & Related papers  (2022-12-14T15:22:13Z)
• Dynamic Spatio-Temporal Specialization Learning for Fine-Grained Action Recognition [19.562218963941227]
  We derive inspiration from the human visual system which contains specialized regions that are dedicated towards handling specific tasks. We design a novel Dynamic Dynamic Spatio-Temporal subset (DSTS) module, which consists of specialized neurons that are only activated for a subset of samples that are highly similar. We design an UpstreamDownstream Learning algorithm to optimize our model's dynamic decisions during training, improving the performance of our DSTS module.
  arXiv  Detail & Related papers  (2022-09-03T13:59:49Z)
• Dynamic Prototype Mask for Occluded Person Re-Identification [88.7782299372656]
  Existing methods mainly address this issue by employing body clues provided by an extra network to distinguish the visible part. We propose a novel Dynamic Prototype Mask (DPM) based on two self-evident prior knowledge. Under this condition, the occluded representation could be well aligned in a selected subspace spontaneously.
  arXiv  Detail & Related papers  (2022-07-19T03:31:13Z)
• Hybrid Routing Transformer for Zero-Shot Learning [83.64532548391]
  This paper presents a novel transformer encoder-decoder model, called hybrid routing transformer (HRT) We embed an active attention, which is constructed by both the bottom-up and the top-down dynamic routing pathways to generate the attribute-aligned visual feature. While in HRT decoder, we use static routing to calculate the correlation among the attribute-aligned visual features, the corresponding attribute semantics, and the class attribute vectors to generate the final class label predictions.
  arXiv  Detail & Related papers  (2022-03-29T07:55:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.