Dissecting Human Body Representations in Deep Networks Trained for Person Identification

• URL: http://arxiv.org/abs/2502.15934v1
• Date: Fri, 21 Feb 2025 21:03:05 GMT
• Title: Dissecting Human Body Representations in Deep Networks Trained for Person Identification
• Authors: Thomas M Metz, Matthew Q Hill, Blake Myers, Veda Nandan Gandi, Rahul Chilakapati, Alice J O'Toole,
• Abstract summary: We analyze body image embeddings from four body identification networks trained with 1.9 million images across 4,788 identities and 9 databases.<n>We show that the face contributes to the accuracy of body identification algorithms and that these algorithms can identify faces to some extent -- with no explicit face training.<n>We demonstrate that identification accuracy can be improved without additional training by operating directly and selectively on the learned embedding space.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long-term body identification algorithms have emerged recently with the increased availability of high-quality training data. We seek to fill knowledge gaps about these models by analyzing body image embeddings from four body identification networks trained with 1.9 million images across 4,788 identities and 9 databases. By analyzing a diverse range of architectures (ViT, SWIN-ViT, CNN, and linguistically primed CNN), we first show that the face contributes to the accuracy of body identification algorithms and that these algorithms can identify faces to some extent -- with no explicit face training. Second, we show that representations (embeddings) generated by body identification algorithms encode information about gender, as well as image-based information including view (yaw) and even the dataset from which the image originated. Third, we demonstrate that identification accuracy can be improved without additional training by operating directly and selectively on the learned embedding space. Leveraging principal component analysis (PCA), identity comparisons were consistently more accurate in subspaces that eliminated dimensions that explained large amounts of variance. These three findings were surprisingly consistent across architectures and test datasets. This work represents the first analysis of body representations produced by long-term re-identification networks trained on challenging unconstrained datasets.

Related papers

• Unconstrained Body Recognition at Altitude and Range: Comparing Four Approaches [0.0]
  We focus on learning persistent body shape characteristics that remain stable over time.<n>We introduce a body identification model based on a Vision Transformer (ViT) and on a Swin-ViT model.<n>All models are trained on a large and diverse dataset of over 1.9 million images of approximately 5k identities across 9 databases.
  arXiv  Detail & Related papers  (2025-02-10T23:49:06Z)
• LightFFDNets: Lightweight Convolutional Neural Networks for Rapid Facial Forgery Detection [0.0]
  This study focuses on image processing-based forgery detection using Fake-Vs-Real-Faces (Hard) [10] and 140k Real and Fake Faces [61] data sets. Two lightweight deep learning models are proposed to conduct forgery detection using these images. It's shown that the proposed lightweight deep learning models detect forgeries of facial imagery accurately, and computationally efficiently.
  arXiv  Detail & Related papers  (2024-11-18T18:44:10Z)
• Analyzing the Impact of Shape & Context on the Face Recognition Performance of Deep Networks [2.0099255688059907]
  We analyze how changing the underlying 3D shape of the base identity in face images can distort their overall appearance. Our experiments demonstrate the significance of facial shape in accurate face matching and underpin the importance of contextual data for network training.
  arXiv  Detail & Related papers  (2022-08-05T05:32:07Z)
• Finding Facial Forgery Artifacts with Parts-Based Detectors [73.08584805913813]
  We design a series of forgery detection systems that each focus on one individual part of the face. We use these detectors to perform detailed empirical analysis on the FaceForensics++, Celeb-DF, and Facebook Deepfake Detection Challenge datasets.
  arXiv  Detail & Related papers  (2021-09-21T16:18:45Z)
• A 3D GAN for Improved Large-pose Facial Recognition [3.791440300377753]
  Facial recognition using deep convolutional neural networks relies on the availability of large datasets of face images. Recent studies have shown that current methods of disentangling pose from identity are inadequate. In this work we incorporate a 3D morphable model into the generator of a GAN in order to learn a nonlinear texture model from in-the-wild images. This allows generation of new, synthetic identities, and manipulation of pose, illumination and expression without compromising the identity.
  arXiv  Detail & Related papers  (2020-12-18T22:41:15Z)
• Continuous Emotion Recognition with Spatiotemporal Convolutional Neural Networks [82.54695985117783]
  We investigate the suitability of state-of-the-art deep learning architectures for continuous emotion recognition using long video sequences captured in-the-wild. We have developed and evaluated convolutional recurrent neural networks combining 2D-CNNs and long short term-memory units, and inflated 3D-CNN models, which are built by inflating the weights of a pre-trained 2D-CNN model during fine-tuning.
  arXiv  Detail & Related papers  (2020-11-18T13:42:05Z)
• Generalized Iris Presentation Attack Detection Algorithm under Cross-Database Settings [63.90855798947425]
  Presentation attacks pose major challenges to most of the biometric modalities. We propose a generalized deep learning-based presentation attack detection network, MVANet. It is inspired by the simplicity and success of hybrid algorithm or fusion of multiple detection networks.
  arXiv  Detail & Related papers  (2020-10-25T22:42:27Z)
• Person image generation with semantic attention network for person re-identification [9.30413920076019]
  We propose a novel person pose-guided image generation method, which is called the semantic attention network. The network consists of several semantic attention blocks, where each block attends to preserve and update the pose code and the clothing textures. Compared with other methods, our network can characterize better body shape and keep clothing attributes, simultaneously.
  arXiv  Detail & Related papers  (2020-08-18T12:18:51Z)
• Structured Landmark Detection via Topology-Adapting Deep Graph Learning [75.20602712947016]
  We present a new topology-adapting deep graph learning approach for accurate anatomical facial and medical landmark detection. The proposed method constructs graph signals leveraging both local image features and global shape features. Experiments are conducted on three public facial image datasets (WFLW, 300W, and COFW-68) as well as three real-world X-ray medical datasets (Cephalometric (public), Hand and Pelvis)
  arXiv  Detail & Related papers  (2020-04-17T11:55:03Z)
• DotFAN: A Domain-transferred Face Augmentation Network for Pose and Illumination Invariant Face Recognition [94.96686189033869]
  We propose a 3D model-assisted domain-transferred face augmentation network (DotFAN) DotFAN can generate a series of variants of an input face based on the knowledge distilled from existing rich face datasets collected from other domains. Experiments show that DotFAN is beneficial for augmenting small face datasets to improve their within-class diversity.
  arXiv  Detail & Related papers  (2020-02-23T08:16:34Z)
• Investigating the Impact of Inclusion in Face Recognition Training Data on Individual Face Identification [93.5538147928669]
  We audit ArcFace, a state-of-the-art, open source face recognition system, in a large-scale face identification experiment with more than one million distractor images. We find a Rank-1 face identification accuracy of 79.71% for individuals present in the model's training data and an accuracy of 75.73% for those not present.
  arXiv  Detail & Related papers  (2020-01-09T15:50:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.