TFPose: Direct Human Pose Estimation with Transformers
- URL: http://arxiv.org/abs/2103.15320v1
- Date: Mon, 29 Mar 2021 04:18:54 GMT
- Title: TFPose: Direct Human Pose Estimation with Transformers
- Authors: Weian Mao, Yongtao Ge, Chunhua Shen, Zhi Tian, Xinlong Wang, Zhibin
Wang
- Abstract summary: We formulate the pose estimation task into a sequence prediction problem that can effectively be solved by transformers.
Our framework is simple and direct, bypassing the drawbacks of the heatmap-based pose estimation.
Experiments on the MS-COCO and MPII datasets demonstrate that our method can significantly improve the state-of-the-art of regression-based pose estimation.
- Score: 83.03424247905869
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: We propose a human pose estimation framework that solves the task in the
regression-based fashion. Unlike previous regression-based methods, which often
fall behind those state-of-the-art methods, we formulate the pose estimation
task into a sequence prediction problem that can effectively be solved by
transformers. Our framework is simple and direct, bypassing the drawbacks of
the heatmap-based pose estimation. Moreover, with the attention mechanism in
transformers, our proposed framework is able to adaptively attend to the
features most relevant to the target keypoints, which largely overcomes the
feature misalignment issue of previous regression-based methods and
considerably improves the performance. Importantly, our framework can
inherently take advantages of the structured relationship between keypoints.
Experiments on the MS-COCO and MPII datasets demonstrate that our method can
significantly improve the state-of-the-art of regression-based pose estimation
and perform comparably with the best heatmap-based pose estimation methods.
Related papers
- A Bayesian Robust Regression Method for Corrupted Data Reconstruction [5.298637115178182]
We develop an effective robust regression method that can resist adaptive adversarial attacks.
First, we propose the novel TRIP (hard Thresholding approach to Robust regression with sImple Prior) algorithm.
We then use the idea of Bayesian reweighting to construct the more robust BRHT (robust Bayesian Reweighting regression via Hard Thresholding) algorithm.
arXiv Detail & Related papers (2022-12-24T17:25:53Z) - Poseur: Direct Human Pose Regression with Transformers [119.79232258661995]
We propose a direct, regression-based approach to 2D human pose estimation from single images.
Our framework is end-to-end differentiable, and naturally learns to exploit the dependencies between keypoints.
Ours is the first regression-based approach to perform favorably compared to the best heatmap-based pose estimation methods.
arXiv Detail & Related papers (2022-01-19T04:31:57Z) - Dynamic Iterative Refinement for Efficient 3D Hand Pose Estimation [87.54604263202941]
We propose a tiny deep neural network of which partial layers are iteratively exploited for refining its previous estimations.
We employ learned gating criteria to decide whether to exit from the weight-sharing loop, allowing per-sample adaptation in our model.
Our method consistently outperforms state-of-the-art 2D/3D hand pose estimation approaches in terms of both accuracy and efficiency for widely used benchmarks.
arXiv Detail & Related papers (2021-11-11T23:31:34Z) - Improving Robustness for Pose Estimation via Stable Heatmap Regression [19.108116394510258]
A heatmap regression method is proposed to alleviate network vulnerability to small perturbations.
A maximum stability training loss is used to simplify the optimization difficulty.
The proposed method achieves a significant advance in robustness over state-of-the-art approaches on two benchmark datasets.
arXiv Detail & Related papers (2021-05-08T03:07:05Z) - Pose Guided Person Image Generation with Hidden p-Norm Regression [113.41144529452663]
We propose a novel approach to solve the pose guided person image generation task.
Our method estimates a pose-invariant feature matrix for each identity, and uses it to predict the target appearance conditioned on the target pose.
Our method yields competitive performance in all the aforementioned variant scenarios.
arXiv Detail & Related papers (2021-02-19T17:03:54Z) - REDE: End-to-end Object 6D Pose Robust Estimation Using Differentiable
Outliers Elimination [15.736699709454857]
We propose REDE, a novel end-to-end object pose estimator using RGB-D data.
We also propose a differentiable outliers elimination method that regresses the candidate result and the confidence simultaneously.
The experimental results on three benchmark datasets show that REDE slightly outperforms the state-of-the-art approaches.
arXiv Detail & Related papers (2020-10-24T06:45:39Z) - Deep Keypoint-Based Camera Pose Estimation with Geometric Constraints [80.60538408386016]
Estimating relative camera poses from consecutive frames is a fundamental problem in visual odometry.
We propose an end-to-end trainable framework consisting of learnable modules for detection, feature extraction, matching and outlier rejection.
arXiv Detail & Related papers (2020-07-29T21:41:31Z) - Bottom-Up Human Pose Estimation by Ranking Heatmap-Guided Adaptive
Keypoint Estimates [76.51095823248104]
We present several schemes that are rarely or unthoroughly studied before for improving keypoint detection and grouping (keypoint regression) performance.
First, we exploit the keypoint heatmaps for pixel-wise keypoint regression instead of separating them for improving keypoint regression.
Second, we adopt a pixel-wise spatial transformer network to learn adaptive representations for handling the scale and orientation variance.
Third, we present a joint shape and heatvalue scoring scheme to promote the estimated poses that are more likely to be true poses.
arXiv Detail & Related papers (2020-06-28T01:14:59Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.