Self-Supervision and Spatial-Sequential Attention Based Loss for Multi-Person Pose Estimation

• URL: http://arxiv.org/abs/2110.10734v1
• Date: Wed, 20 Oct 2021 19:13:17 GMT
• Title: Self-Supervision and Spatial-Sequential Attention Based Loss for Multi-Person Pose Estimation
• Authors: Haiyang Liu, Dingli Luo, Songlin Du, Takeshi Ikenaga
• Abstract summary: Bottom-up based pose estimation approaches use heatmaps with auxiliary predictions to estimate joint positions and belonging at one time. The lack of more explicit supervision results in low features utilization and contradictions between predictions in one model. This paper proposes a new loss organization method which uses self-supervised heatmaps to reduce prediction contradictions and spatial-sequential attention to enhance networks' features extraction.
• Score: 6.92027612631023
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Bottom-up based multi-person pose estimation approaches use heatmaps with auxiliary predictions to estimate joint positions and belonging at one time. Recently, various combinations between auxiliary predictions and heatmaps have been proposed for higher performance, these predictions are supervised by the corresponding L2 loss function directly. However, the lack of more explicit supervision results in low features utilization and contradictions between predictions in one model. To solve these problems, this paper proposes (i) a new loss organization method which uses self-supervised heatmaps to reduce prediction contradictions and spatial-sequential attention to enhance networks' features extraction; (ii) a new combination of predictions composed by heatmaps, Part Affinity Fields (PAFs) and our block-inside offsets to fix pixel-level joints positions and further demonstrates the effectiveness of proposed loss function. Experiments are conducted on the MS COCO keypoint dataset and adopting OpenPose as the baseline model. Our method outperforms the baseline overall. On the COCO verification dataset, the mAP of OpenPose trained with our proposals outperforms the OpenPose baseline by over 5.5%.

Related papers

• Spectral Representation for Causal Estimation with Hidden Confounders [33.148766692274215]
  We address the problem of causal effect estimation where hidden confounders are present. Our approach uses a singular value decomposition of a conditional expectation operator, followed by a saddle-point optimization problem.
  arXiv  Detail & Related papers  (2024-07-15T05:39:56Z)
• Hierarchical Forecasting at Scale [55.658563862299495]
  Existing hierarchical forecasting techniques scale poorly when the number of time series increases. We propose to learn a coherent forecast for millions of time series with a single bottom-level forecast model. We implement our sparse hierarchical loss function within an existing forecasting model at bol, a large European e-commerce platform.
  arXiv  Detail & Related papers  (2023-10-19T15:06:31Z)
• LoLep: Single-View View Synthesis with Locally-Learned Planes and Self-Attention Occlusion Inference [66.45326873274908]
  We propose a novel method, LoLep, which regresses Locally-Learned planes from a single RGB image to represent scenes accurately. Compared to MINE, our approach has an LPIPS reduction of 4.8%-9.0% and an RV reduction of 73.9%-83.5%.
  arXiv  Detail & Related papers  (2023-07-23T03:38:55Z)
• Off-Policy Evaluation for Large Action Spaces via Conjunct Effect Modeling [30.835774920236872]
  We study off-policy evaluation of contextual bandit policies for large discrete action spaces. We propose a new estimator, called OffCEM, that is based on the conjunct effect model (CEM), a novel decomposition of the causal effect into a cluster effect and a residual effect. Experiments demonstrate that OffCEM provides substantial improvements in OPE especially in the presence of many actions.
  arXiv  Detail & Related papers  (2023-05-14T04:16:40Z)
• Forecast reconciliation for vaccine supply chain optimization [61.13962963550403]
  Vaccine supply chain optimization can benefit from hierarchical time series forecasting. Forecasts of different hierarchy levels become incoherent when higher levels do not match the sum of the lower levels forecasts. We tackle the vaccine sale forecasting problem by modeling sales data from GSK between 2010 and 2021 as a hierarchical time series.
  arXiv  Detail & Related papers  (2023-05-02T14:34:34Z)
• Patch-level Gaze Distribution Prediction for Gaze Following [49.93340533068501]
  We introduce the patch distribution prediction ( PDP) method for gaze following training. We show that our model regularizes the MSE loss by predicting better heatmap distributions on images with larger annotation variances. Experiments show that our model bridging the gap between the target prediction and in/out prediction subtasks, showing a significant improvement on both subtasks on public gaze following datasets.
  arXiv  Detail & Related papers  (2022-11-20T19:25:15Z)
• Heatmap Distribution Matching for Human Pose Estimation [12.524484316923333]
  We show that optimizing the heatmap prediction in such a way, the model performance of body joint localization may not be consistently improved. We propose to formulate the optimization of the heatmap prediction as a distribution matching problem between the predicted heatmap and the dot annotation of the body joint. We show the effectiveness of our proposed method through extensive experiments on the COCO dataset and the MPII dataset.
  arXiv  Detail & Related papers  (2022-10-03T07:21:30Z)
• Deconfounding Scores: Feature Representations for Causal Effect Estimation with Weak Overlap [140.98628848491146]
  We introduce deconfounding scores, which induce better overlap without biasing the target of estimation. We show that deconfounding scores satisfy a zero-covariance condition that is identifiable in observed data. In particular, we show that this technique could be an attractive alternative to standard regularizations.
  arXiv  Detail & Related papers  (2021-04-12T18:50:11Z)
• Supervised PCA: A Multiobjective Approach [70.99924195791532]
  Methods for supervised principal component analysis (SPCA) We propose a new method for SPCA that addresses both of these objectives jointly. Our approach accommodates arbitrary supervised learning losses and, through a statistical reformulation, provides a novel low-rank extension of generalized linear models.
  arXiv  Detail & Related papers  (2020-11-10T18:46:58Z)
• DeeSCo: Deep heterogeneous ensemble with Stochastic Combinatory loss for gaze estimation [7.09232719022402]
  We introduce a deep, end-to-end trainable ensemble of heatmap-based weak predictors for 2D/3D gaze estimation. We show that our ensemble outperforms state-of-the-art approaches for 2D/3D gaze estimation on multiple datasets.
  arXiv  Detail & Related papers  (2020-04-15T14:06:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.