TransPose: Towards Explainable Human Pose Estimation by Transformer

• URL: http://arxiv.org/abs/2012.14214v2
• Date: Thu, 31 Dec 2020 07:15:16 GMT
• Title: TransPose: Towards Explainable Human Pose Estimation by Transformer
• Authors: Sen Yang and Zhibin Quan and Mu Nie and Wankou Yang
• Abstract summary: We construct an explainable model named TransPose based on Transformer architecture and low-level convolutional blocks. Given an image, the attention layers built in Transformer can capture long-range spatial relationships between keypoints. Experiments show that TransPose can accurately predict the positions of keypoints.
• Score: 17.39838556906491
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep Convolutional Neural Networks (CNNs) have made remarkable progress on human pose estimation task. However, there is no explicit understanding of how the locations of body keypoints are predicted by CNN, and it is also unknown what spatial dependency relationships between structural variables are learned in the model. To explore these questions, we construct an explainable model named TransPose based on Transformer architecture and low-level convolutional blocks. Given an image, the attention layers built in Transformer can capture long-range spatial relationships between keypoints and explain what dependencies the predicted keypoints locations highly rely on. We analyze the rationality of using attention as the explanation to reveal the spatial dependencies in this task. The revealed dependencies are image-specific and variable for different keypoint types, layer depths, or trained models. The experiments show that TransPose can accurately predict the positions of keypoints. It achieves state-of-the-art performance on COCO dataset, while being more interpretable, lightweight, and efficient than mainstream fully convolutional architectures.

Related papers

• DAPE V2: Process Attention Score as Feature Map for Length Extrapolation [63.87956583202729]
  We conceptualize attention as a feature map and apply the convolution operator to mimic the processing methods in computer vision. The novel insight, which can be adapted to various attention-related models, reveals that the current Transformer architecture has the potential for further evolution.
  arXiv  Detail & Related papers  (2024-10-07T07:21:49Z)
• How Deep Networks Learn Sparse and Hierarchical Data: the Sparse Random Hierarchy Model [4.215221129670858]
  We show that by introducing sparsity to generative hierarchical models of data, the task acquires insensitivity to spatial transformations that are discrete versions of smooth transformations. We quantify how the sample complexity of CNNs learning the SRHM depends on both the sparsity and hierarchical structure of the task.
  arXiv  Detail & Related papers  (2024-04-16T17:01:27Z)
• On Characterizing the Evolution of Embedding Space of Neural Networks using Algebraic Topology [9.537910170141467]
  We study how the topology of feature embedding space changes as it passes through the layers of a well-trained deep neural network (DNN) through Betti numbers. We demonstrate that as depth increases, a topologically complicated dataset is transformed into a simple one, resulting in Betti numbers attaining their lowest possible value.
  arXiv  Detail & Related papers  (2023-11-08T10:45:12Z)
• Capsules as viewpoint learners for human pose estimation [4.246061945756033]
  We show how most neural networks are not able to generalize well when the camera is subject to significant viewpoint changes. We propose a novel end-to-end viewpoint-equivariant capsule autoencoder that employs a fast Variational Bayes routing and matrix capsules. We achieve state-of-the-art results for multiple tasks and datasets while retaining other desirable properties.
  arXiv  Detail & Related papers  (2023-02-13T09:01:46Z)
• Robust Change Detection Based on Neural Descriptor Fields [53.111397800478294]
  We develop an object-level online change detection approach that is robust to partially overlapping observations and noisy localization results. By associating objects via shape code similarity and comparing local object-neighbor spatial layout, our proposed approach demonstrates robustness to low observation overlap and localization noises.
  arXiv  Detail & Related papers  (2022-08-01T17:45:36Z)
• BTranspose: Bottleneck Transformers for Human Pose Estimation with Self-Supervised Pre-Training [0.304585143845864]
  In this paper, we consider the recently proposed Bottleneck Transformers, which combine CNN and multi-head self attention (MHSA) layers effectively. We consider different backbone architectures and pre-train them using the DINO self-supervised learning method. Experiments show that our model achieves an AP of 76.4, which is competitive with other methods such as [1] and has fewer network parameters.
  arXiv  Detail & Related papers  (2022-04-21T15:45:05Z)
• DepthFormer: Exploiting Long-Range Correlation and Local Information for Accurate Monocular Depth Estimation [50.08080424613603]
  Long-range correlation is essential for accurate monocular depth estimation. We propose to leverage the Transformer to model this global context with an effective attention mechanism. Our proposed model, termed DepthFormer, surpasses state-of-the-art monocular depth estimation methods with prominent margins.
  arXiv  Detail & Related papers  (2022-03-27T05:03:56Z)
• Swin-Pose: Swin Transformer Based Human Pose Estimation [16.247836509380026]
  Convolutional neural networks (CNNs) have been widely utilized in many computer vision tasks. CNNs have a fixed reception field and lack the ability of long-range perception, which is crucial to human pose estimation. We propose a novel model based on transformer architecture, enhanced with a feature pyramid fusion structure.
  arXiv  Detail & Related papers  (2022-01-19T02:15:26Z)
• PnP-DETR: Towards Efficient Visual Analysis with Transformers [146.55679348493587]
  Recently, DETR pioneered the solution vision tasks with transformers, it directly translates the image feature map into the object result. Recent transformer-based image recognition model andTT show consistent efficiency gain.
  arXiv  Detail & Related papers  (2021-09-15T01:10:30Z)
• Transformers Solve the Limited Receptive Field for Monocular Depth Prediction [82.90445525977904]
  We propose TransDepth, an architecture which benefits from both convolutional neural networks and transformers. This is the first paper which applies transformers into pixel-wise prediction problems involving continuous labels.
  arXiv  Detail & Related papers  (2021-03-22T18:00:13Z)
• Spatio-Temporal Inception Graph Convolutional Networks for Skeleton-Based Action Recognition [126.51241919472356]
  We design a simple and highly modularized graph convolutional network architecture for skeleton-based action recognition. Our network is constructed by repeating a building block that aggregates multi-granularity information from both the spatial and temporal paths.
  arXiv  Detail & Related papers  (2020-11-26T14:43:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.